1.1.3 Neural Networks and Deep Learning

 Neural network algorithms are a class of algorithms that learn from data based on neyural networks. They still belong to the category of machine learning. Due to the limitation of computing power and data volume, early neural networks were shallow, usually with around one to four layers. Therefore, the network expression ability was limited. With the improvement of computing power and the arrival of the big data era, highly parallelized graophics processing units(GPUs) and massive data make training of large-scale neural networks possible.

In 2006, Geoffrey Hinton first proposed the concept of deep learning. In 2012, AlexNet, and eight-layer deep nerural network, was released and achieved huge performance imprevements in the image recongnition competition. Since then, neural network models with dozens, hundreds, and even thousands of layers have been developed successively, showing strong learning ability. Algorithms implemented using deep neural networks are generallty referred to as deep learning models. In essence, neural networks and deep learning can be considered the same.

Let's simply compare deep learning with other algorithms. As shown in Figure 103, rule-baed systems usually write explicit logic, which is generally designed for specific tasks and is not suitable for other tasks. Traditional machine learning algorithms artificially design feature detection methods with certain generality, such as SIFT nad HOG features. These featureas are suitable for a certain type of tasks and have certain generality. but the performance highly depends onb how to desing those features. The emergence of neural hetworks ahs made it possible for omputers to design those features automatically through nerual nwtworks without human intervention. Shallow nerual networks typically have limited feature extraction capability, while deep neural networks are capable of extracting high-level, abstract features and have better performance.