Deep Learning - Tijn van der Zant

Publications, books and careful research online can provide a wealth of information about Artificial Intelligence and how it differs from Machine Learning and Deep Learning.

Nevertheless, laymen and even the media often get these three related but distinct concepts confused. In this part of a special series of articles by SIM-CI, Tijn van der Zant, Artificial Intelligence Lead at SIM-CI, talks about what Deep Learning is, his personal experience as a pioneering Deep Learning researcher, and the tremendous impact this technology has on our world now and in the future.

ON DEEP LEARNING Tijn van der Zant has no trouble explaining Artificial Intelligence to anyone who asks. “Sometimes I explain what I'm doing is: if you see (artificial) intelligence in science fiction movies, well that’s what I’m working on.” He goes on to say that, foremost, Artificial Intelligence is quite broad. “It is the encompassing field, there's a lot of stuff in AI.” One of those subcategories in AI is Machine Learning. Machine Learning involves processes that require relatively large amounts of data and algorithm to predict specific outcomes, such as recognising spam mail or predicting traffic flow, for instance: in short, processes that are quite smart but not without limitations. Deep Learning, on the other hand, is a subcategory of Machine Learning but one which goes practically “deeper.”

To illustrate an example of Deep Learning, Tijn recalls a brief from a client this year, specifically a supplier for car factories, requiring what was called “error-detection” in car paints, error in this case meaning scratches, dents or scuffs. Put simply, Deep Learning algorithms can “see” and “learn” these irregularities and identify them which in turn would inform as to the best repair option. “I got data gathered from a couple of hundred cars, in the form of three-fold scans which when combined would result into a recognisable image of the affected surface.” Using ocular inspection alone, people would mislabel these quite often, leading to an error margin of 20%. This margin of error adversely affects the decision-making of which reparative option should be applied. The entire project was to last three months. Implementing Deep Learning on the data took only three days and reduced the error margin to 3%, an impressive 85% improvement.

That said, the process wasn’t without a hitch. Two of those three months were devoted to what he calls “debugging the human component.” Tijn says, “For one, because of shortage of digital storage space, the company deleted all the original data.” Not only that, the images he was given have already been tampered with using a software to make them look ‘nicer.’ This disrupted the process in such a way that he spent several weeks unsuccessfully applying the algorithm. Fortunately, after a thorough enquiry, the one who processed the images managed to produce the raw originals from his hard drive, and Tijn could proceed and complete his brief.

It is this deeply inherent data-driven quality of Deep Learning which requires heavy emphasis on meticulous data-gathering. Seasoned Deep Learning experts know that as much as two-thirds of time can be spent on “pre-processing” the data, and would know best to lessen or avoid this timeconsuming stage.

ON CRITICAL INFRASTRUCTURE AND THE DEMAND FOR NEW EXPERTS According to Tijn, it’s a four-step process of virtualisation and digitisation that is key to approach different Deep Learning applications. “First, make the system electronic. Second, make sure you have great, powerful software. Third, run that software in the Cloud. And fourth, use AI” This is being applied by many existing companies already, from Tesla, Google, Amazon to SIM-CI, and is perfectly applicable to more. “It's completely normal for me to think of a robotic system that slips into the sewer or into the water main or gas pipe dragging along some wires which it then attaches with sensors to the insides of the pipe so that it can be monitored and measured continuously,” Tijn says.

The issue, though, is the reluctance of agencies and DSOs to adopt technology outside of conservative options, which can be decades out of step with the industry’s needs and with advancements in technology. “There’s a reason companies like SIM-CI are popping up. They aren’t stuck in the system, can think out of the box and are capable of moving in different directions.”

There are much too few entities in the Netherlands adopting Deep Learning and there is a more widespread reason why this is so. “We need to train people first. Talent is short, and yes, you need very bright people. Two weeks ago I heard that in the US alone, there is a shortage of 1.5 million data scientists. I made an estimate for the entire planet and it’s short of about five to eight million data scientists in all. Not all of them will become Deep Learning experts, but even if only 10% of those people do apply it, where do you find those people?” he says. Luckily, the Netherlands is the only country that offers a curriculum for AI at undergraduate, graduate and post-graduate levels, yet that’s not yet the case for Deep Learning. “When it comes to Deep Learning you have to know exactly what you are doing. But the problem is it was only two or three years ago that the first PhD programmes started. We have people now with as much years (2-3) experience in it. It’s a very young field, and we need way more people.”

ON EXPERTS LEARNING FROM ONE ANOTHER “The most important thing that you have to do is find research that already tackled a similar kind of problem as yours, find those related papers, read them, figure out what was working and what was not working, and then use this to implement your own architecture. To be able to do that you need a PhD level of Deep Learning knowledge.” Tijn cannot stress enough the importance of checking current studies into the field. “There is a kind of publish-or-perish culture in science. One needs to publish lots of papers or otherwise you’re fired. Since confirming somebody else's research is not something you can write a paper about, researchers barely check each other's works anymore. As a result, during implementation, we would now and again come across formulas that don’t seem to make sense.”

“In 2007 I applied the visual cortex of primate brains as a standard model in handwriting recognition.

It required me to start a computation that would last at least three and a half months on 200 computers. Contrary to others who doubted me, my professor believed it might work. When the computation time was over, we ran it on my professor’s data and my model in fact beat his best model. Even though he was a sort of ‘grandmaster’ of handwriting recognition, the professor’s

brilliant reaction was to immediately say ‘Wow, this works, I’m going to apply this model in my own research!’” This man, Tijn believes, personifies the real scientists who push progress forward by building on working techniques and doing something further with them.

ON THE REVOLUTIONARY IMPACT OF DEEP LEARNING The 1990s saw the so-called Fourth Wave of neural networks having seen the previous incarnations of what essentially are types of architecture getting incrementally more and more sophisticated. Tijn says, “Those were already very interesting times because researchers were experimenting with all sorts of different architectures. But progress was hampered because the computing hardware was slow. “Back in 2007, that handwriting recognition model mentioned earlier was actually one of the earlier Deep Learning researches but there was no name for it yet back then of course.” These days, however, there are specialised hardware with something called tenths of processing unit (TPU) and this is speeding up Deep Learning 15 to 20 times. “Basically these numbers mean it’s like you’re being morphed into the future,” he says.

And as Deep Learning speeds up at an incredible rate as we speak, Tijn sees many industries and fields being completely transformed. “I hope there will soon be a government that will actually use the data collated from all the different things that they’ve been doing the past decade and then apply Machine Learning or Deep Learning to those and figure out which policies actually work. Using this fact-based decision process as opposed to a politically-motivated one could potentially save us tens of billions of euros a year.”

Tijn’s own company Robolect is involved in several projects and one of them is for financial technology. “What we're building would replace 80% of the tasks that accountants have to do,” he says. The financial industry in general is also going to be completely be disrupted. “In finance we know in five to ten year’s time possibly 50% to even up to 80% of the people will be jobless. I’m amazed that we are still training as many people in finance as we do, acting like it’s the 1980s.” Tijn also thinks the justice system would be impacted. “Lawyers have to do an awful lot of research to look at basically similar cases and even then the results are inconsistent. Researchers are working on a system that might actually replace a judge. Outside exceptionally complex court cases, it will be fed all the facts of a simple case and one quickly gets a decision.” Health management is another field on which Deep Learning will have major industry-changing influence. “I’ve already seen projects where they use Deep Learning on MRI scans to detect areas where there may be cancerous tumours, for example. Non-Deep Learning applications also exist, I know, but these non-Deep Learning applications also use Machine Learning that takes only a few minutes whereas a specialist normally would take an hour to an hour and a half.”

These are but a few areas in which Deep Learning clearly have applications. “What should also happen simultaneously is a discussion on EU or worldwide level about what we should do about these millions of people who would be left without jobs,” according to Tijn. A switch in economic and labour policies is crucial and inevitable. “It's changing already and it will only change faster and there are some predictions that say that about half of the jobs will be replaced by 2025.”

ON THE FUTURE AND IMPROVING THE WORLD “In 2006 I began RoboCup@Home, a robotic benchmark designed to assist in building robots for elderly care. These benchmarks—such as fetching something from the refrigerator or recognising if a human is doing a yoga pose as opposed to having a heart attack on the floor, among many others— focus on what we ‘cannot’ do. As we still encounter year after year many of these goals we’ve not yet achieved, this in turn guides the research internationally among 250 participating laboratories worldwide and spending 100 to 150 million euros a year designing robots for elderly care.” It is imperative that we build intelligent systems to help us, according to Tijn, to ultimately improve lives preferably without detriment to our planet. “If every human being on this planet aspires to have the standard of living of a European, North American, or East Asian, we need four or five planets at the rate we’re doing it. And we don’t have those, so we have to come up with completely radical new solutions because otherwise we won’t make it to the 22nd century on this planet.”

When asked about his optimism about the future, Tijn is not certain. “Despite technology becoming more and more powerful, we are still capable of doing quite nasty things as a species. It will be a race between being being able to colonise another planet or ultimate self-destruction and I’m not sure which one will win.” But in the case of future technology, he is quite clear. “AI capabilities which six years ago we thought won’t arrive until 2025 or 2030 are in fact here now. Even experts who work in the field are in some sort of future shock. We are all very much exploring what is possible so I think we’ve seen nothing yet.”

Tijn van der Zant, AI Lead SIM-CI In 2007 he used models of the visual cortex for handwriting recognition, pioneering the field of deep learning. Tijn worked on the first cognitive benchmarks for robots which are now an integral part of RoboCup@Home. Since 2010 he is commercially active solving complex A.I. and machine learning problems for companies ranging from large scale text classification, deep learning on industrial data, assisting car manufacturers with the development of autonomously driving vehicles and solving complex FinTech problems.

Tijn van der Zant is also an advisory board member of the Robotics/AI Board of the Lifeboat Foundation, a non-profit nongovernmental organization dedicated to encouraging scientific advancements while helping humanity survive existential risks and possible misuse of increasingly powerful technologies.

Interview, text and infographics by: Peter Aquino