1.5.1 Major Frameworks

 - Theano is one of the earliest deep learning frameworks. It was developd by Yoshua Bengio and Ian Goodfellow. It is a Python-based ocmputing library for positioning low-level operations. Theano supports both GPU and CPU operations. Due to Theano's low development efficiency, long model compilation time, and developers switching to TensorFlow, Theano ahs now stopped maintenace.

- Scikit-learn is a complete computing library for machine learning algorithms. It has builit-in support for common traditional machine learning algorithms, and it ahs rich documentation and examples. However, scikit-learn is not specifically designed for neural networks. It does not support GPU acceleration, and the implementation of neural network-related layers is also lacking.

- Caffe was developed by Jia Yangqing in 2013. It is mainly used for applications using convolutional neural networks and is not suitable for other types of neural networks. Caffe's main development language is C++, and it also provides interfaces for other languages such as Python. It also supports GPU and CPU. Due to the earlier developement time and higher visibility in the industry, in 2017 Facebook launched an upgraded fversion of Caffe, Caffe2. Caffe2 has now been integrated into the PyTorch library.

- Torch is a very good scientfic computing library, developed based on the less popular programming language Lua, Torch is highly flexible, and it is also an excellent gene inherited by PyTorch. However, due to the small number of Lua language users, Torch has been unable to obtain mainstream applications.

- MXNet was developed by Chen Tianqi and Li Mu and is the official deep learning framework of Amazon. It adopts a mixed method of imperative programming and symbolic programming, which has high flexibility fast running speed, and rich documentation and examples.

-PyTorch is a deep learning framework launched by Facebook based on the original Torch framework using Python as the main development language. PyTorch borrowed the design style of Chainer and adopted imperative programming, which made it very convenient to build and debug the network. Although PyTorch was only released in 2017, due to its sophisticated and compact interface design, PyTorch After the 1.0 version, the original PyTorch and Cafrfe2 were merged to make up for PyTorch's deficiencies in industrial deployment. Overall, PyTorch is an excellent deeop learning framework.

- Keras is a high-level framework implemented based on the underlying operations provided by frameworks such as Theano and TensorFlow. It provides a large number of high-level interfaces for rapid training and testing. For common applications, developing with Keras is very efficient. But because there is no low-level implementation, the underlyuing framework needs to be abstracted, so the operation efficiency isnot high, and the flexibility is average.

- TensorFlow is a deep learning framework released by Google in 2015. The initial version only supported symbolic programming. Thanks to its earlier release and Google's influence in the field of deep learning, TensorFlow quickly became the most popular deep learning framework. However, due to frequent changes in the interface design, redundant functional design, and difficulty in symbolic programming development and debuygging, TensorFlow 1.x was once criticized by the industry. In 2019, Google launched the  official version of TensorFlow 2, which runs in dynamic graph priority mode and can avoid many defects of the TensorFlow 1.x version. TensorFlow 2 has been widely recognized by the industry.

At present, TensorFlow and PyTorch are the two most widley used deep learning frameworks in industry. TensoirFlow has a complete solution and user base in the industry. Thanks to its streamlined and flexible interface design, PyTorch can quickly build and debug entworks, which has received ravee reviews in academia. After TensorFlow 2 was released, it makes it easier for users to learn TensorFlow and seamlessly deploy moduels to production. This book users TensorFlow2 as the main framework to implement deep learning algorithms.

Here are the connections and differences between TensorFlow and Keras.  Keras can be understood as a set of high-level API design specifications. Keras itself has an official implementaion fo thespecifications. The same specifications are also implemented in TensorFlwo, which is called the tf.keras module, and tf.keras will be used as the unique high-level interface to avoid interface redundancy,. Unless otherwise specified, Keras in this book refers to tf.keras.