Within the ML field, there is an area that is often referred to as brain-inspired computation, which is a program aiming to emulate some aspects of how we understand the brain to operate. Since it is believed that the main computational elements a human brain are 86 billion neurons, the two subareas of brain-inspired computation are both inspired by the architecture of a neuron 
, as shown in Figure A.2-1
Compared to spiking computing approaches, e.g. 
, the more popular ML approaches are using "neural network"
as the model. Neural networks (NN) take their inspiration from the notion that a neuron's computation involves a weighted sum of the input values. But instead of simply outputting the weighted sum, a NN applies a nonlinear function to generate an output only if the inputs cross some threshold, as shown in Figure A.2-1
(a). Figure A.2-1
(b) shows a diagrammatic picture of a computational neural network. The neurons in the input layer receive some values and propagate them to the neurons in the middle layer of the network, which is also called a "hidden layer"
. The weighted sums from one or more hidden layers are ultimately propagated to the output layer, which presents the final outputs of the network 
Neural networks having more than three layers, i.e., more than one hidden layer are called deep neural networks (DNN). In contrast to the conventional shallow-structured NN architectures, DNNs, also referred to as deep learning, made amazing breakthroughs since 2010s in many essential application areas because they can achieve human-level accuracy or even exceed human accuracy. Deep learning techniques use supervised and/or unsupervised strategies to automatically learn hierarchical representations in deep architectures for classification 
. With a large number of hidden layers, the superior performance of DNNs comes from its ability to extract high-level features from raw sensory data after using statistical learning over a large amount of data to obtain an effective representation of an input space 
. In recent years, thanks to the big data obtained from the real world, the rapidly increased computation capacity and continuously-evolved algorithms, DNNs have become the most popular ML models for many AI applications.