What Is Deep Learning And How It Works

We enter the field of “deep learning”, deep learning, which is essentially based on artificial neural networks. To follow a logical thread, it is necessary to repeat some concepts already expressed above.

1. First of all, we must remember what a human neuron is: it is a cell that collects and routes nerve impulses; in the social nervous system there are more than 100,000 of them, and they play a vital role because without signal reception and transmission there is no life. So the first step was that made by McCulloch and Pitts in 1943 by assimilating the behaviour of the human neuron to binary computation (presence/absence of signal).
2. The second step was taken by Rosenblatt with his perceptron introducing the first artificial neural network scheme based on an input and an output layer, in the middle and intermediate learning rule. As we have seen, the application of this mathematical model, however, proves incapable of solving complex problems.
3. A decisive step is taken by Rumelhart with the introduction of the third layer of neural networks (what is called hidden) opening the way to MLP networks – Multi-Layers Perceptron. It is in the “hidden” levels that the “magic” occurs and the artificial neural network approaches the human nervous system: in the hidden levels each neuron of a level is connected to all the neurons of the immediately preceding level and to all those of the directly following group; each link has a “weight” (a numerical value) that measures how important the link between two particular neurons.

From a conceptual point of view, this is the point we have arrived at today, and the evolutions are for the moment limited to the invention of increasingly sophisticated algorithms based on this concept. Not that it is a simple thing, mind you, also because the real power of the algorithm is given by the ability to train the neural network, to make it gain experience. Such as? Let’s explain it in 3 steps:

Learning phase: Backpropagation is usually used: an input relating to an exercise of which the network knows the correct outcome is inserted into the network, it is reached at the end by passing it through all the hidden layers; since the system is “ignorant”, the transition from one layer to another occurs randomly, and the output will almost certainly be wrong; knowing, however, what the correct answer should have been, the network understands how unfair it was and goes back along the path by setting different parameters that, at each level, are getting closer and closer to the correct way. The more examples (the better they are) that are “digested” by the network and propagated backwards, the greater the probability that the system will make the right associations arrive at the correct answer.

Test phase: When the programmer believes that the network is sufficiently educated, he enters the system with inputs relating to problems for which he (but not the network) knows the solution. This allows him to understand if the network is ready to be able to process real problems on new data or if he still makes mistakes too often and therefore needs to be better educated. If you get the idea that this is relatively simple, you are off the mark: it takes millions of training and testing sessions to put an effective neural network into production.

Put into production: Once the test phase has been passed, the work does not end with the putting into production of the network. In fact, feedback mechanisms need to be created because a system that works well today does not necessarily work well tomorrow: contexts, behaviours, scenarios change, and the network must be able to update itself in near real-time; to do this she needs mechanisms that make her understand if she is going in the right direction or not.

An important frontier for the evolution of neural networks, in terms of processing capacity, is represented by research in the field of nanotechnologies and in particular of neuromorphic chips.

Also Read: Deep Learning: Concept, Definition & Examples