Integrated Image Sensor and Light Convolutional Neural Network for Image Classification

Deep learning has accomplished huge success in computer vision applications such as self-driving vehicles, facial recognition, and controlling robots. A growing need for deploying systems on resource-limited or resource-constrained environments such as smart cameras, autonomous vehicles, robots, smartphones, and smart wearable devices drives one of the current mainstream developments of convolutional neural networks: reducing model complexity but maintaining fine accuracy. In this study, the proposed efficient light convolutional neural network (ELNet) comprises three convolutional modules which perform ELNet using fewer computations, which is able to be implemented in resource-constrained hardware equipment. The classification task using CIFAR-10 and CIFAR-100 datasets was used to verify the model performance. According to the experimental results, ELNet reached 92.3% and 69%, respectively, in CIFAR-10 and CIFAR-100 datasets; moreover, ELNet effectively lowered the computational complexity and parameters required in comparison with other CNN architectures.


Introduction
Convolutional neural network (CNN) was firstly introduced in the 1980s. At that time, Lecun et al. [1] proposed a simply constructed CNN architecture which contains three convolutional layers, two subsampling layers, and a fully connected layer. LeNet was mainly used for handwriting recognition in the MNIST dataset and obtained the lowest error rate. However, the hardware equipment was not advanced, and graphics processing units had not been invented which led to the development of CNN being greatly restricted. In 2012, Krizhevsky et al. [2] developed AlexNet and won the first place in the ImageNet large-scale visual recognition competition by achieving a top-5 error of 15.3%. Compared with LeNet, AlexNet uses rectified linear unit (ReLU) to replace the conventional sigmoid activation function in order to resolve the vanishing gradient problem. Moreover, the dropout [3] regularization technique was also introduced to reduce overfitting in neural networks. In general, AlexNet extends its network architecture resulting in the requirement of nearly 60 million parameters, and the floating-point operations (FLOPs) have reached 0.7 giga FLOPs. Subsequently, researchers have continued to deepen networks to improve the accuracy such as VGGNet [4].
Instead of deepening the CNN architecture, some researchers expand the width of the network architectures. For instance, Szegedy et al. [5] firstly came up with a concept of inception block in the CNN which encapsulates different sizes of kernels for extracting global and local features. It adjusts the computations by adding a bottleneck layer of a 1 × 1 convolutional filter before applying large-size kernels. Furthermore, Srivastava et al. [6] designed a new architecture to moderate gradient-based training of very deep networks which is called highway network. is network imitates the horizontal expansion concept using the gating function to adaptively bypass the input so that the network can go deeper. In addition, He et al. [7] proposed ResNet by taking inspiration from the bypass and bottleneck layer approaches for reducing the amount of operations. Many improved designs of network architectures are proposed and applied in many applications, such as object detection [8] and semantic analysis [9]. However, regardless of deepening or widening the network architectures, high computational cost and memory requirement are the two main concerns observed with these architectures.
To further alleviate these two primary concerns of the network, designing a lightweight architecture without compromising the performance is necessary, especially when the CNN model is implemented in resource-constrained hardware. Howard et al. [10] adopted depthwise separable convolution in the MobileNet to reduce the model parameters so that the model can be embedded in portable devices for mobile and embedded vision applications. Juefei-Xu et al. [11] proposed the local binary convolutional neural network which adopts local binary convolution (LBC) as a substitute for the conventional CNN. e experimental results showed that the LBC module performs a good approximation of a conventional convolutional layer and results in a major reduction in the number of learnable parameters while training the network. Iandola et al. introduced SqueezeNet [12] which replaces 3 × 3 filters with 1 × 1 filters and decreases the number of input channels to 3 × 3 filters. ese strategies are desirable to decrease the quantity of parameters in a CNN while attempting to maintain accuracy. According to the experimental results reported by Iandola et al., the parameters used in SqueezeNet are 50x fewer than those in AlexNet; besides, it preserves AlexNetlevel accuracy on ImageNet. Others such as parameter pruning and quantization can reduce redundant parameters which reduces the network complexity and addresses the overfitting problem. Furthermore, without decreasing accuracy, more improvements of YOLO were also proposed [13,14] to prove that light CNN can reduce training time and make applications more diverse without being limited by hardware. e three modules provide capabilities and advantages: saving computations when kernel size and the number of kernels are large using depthwise separable convolution, expanding the field of view (FOV) of filters without increasing parameters by atrous convolution, and extracting local and global features simultaneously adopted by the inception module to reduce the parameters and operations of the CNN. In this study, the proposed model, efficient light convolutional neural network (ELNet) with the three modules, is no longer limited by memory and computational constraints. e rest of the paper is organized as follows. In Section 2, the conventional CNN architecture is briefly reviewed. e ELNet is introduced in Section 3. e experimental results using CIFAR-10 and CIFAR-100 datasets are revealed in Section 4 and compared with other state-of-the-art CNN architectures such as GoogLeNet, ResNet-50, and Mobile-Net. Lastly, Section 5 draws conclusions.

Convolutional Neural Network (CNN)
e concept of neural networks mainly comes from biological neural network systems; however, neural networks are connected in a fully connected manner which causes a great amount of calculations when the input size is large. erefore, in the 1980s, convolution kernel was first introduced and then was widely applied in image processing.
ere are four main parts of the CNN: convolutional layer, pooling layer, activation function, and fully connected layer. e function of feature extraction depends on the first three parts, and the fully connected layer is used to classify the obtained features. More descriptions of these parts are explained as follows.

Convolutional Layer.
A convolutional layer consists of a set of learnable filters (or kernels) which have a small receptive field; however, feature extraction can be acquired by extending filters through the full depth of the input volume. e formula is as follows: where r and c represent the row and column of the feature map, n is the number of input channels, k w and k h are the width and height of a convolution kernel, W k i,j is the weight of the i th row and j th column convolution kernel in the k th channel, I k i,j is the input of the i th row and j th column in the k th channel, and b is the bias.

Pooling Layer.
In order to effectively extract features, most of the moving strides are set as 1; yet, this setting causes relatively more operations. erefore, pooling layer is usually added in the CNN for effectively reducing the amount of operations. Equation (2) shows the calculation of max pooling and average pooling: where O r,c is the output row and column, I i,j is the row and column of the input image, and P w and P h are the width and height of the pooling kernel.

Activation Function.
e conventional operation of the convolution kernel is a linear operation; LeNet adopts sigmoid function as an activation function to solve nonlinear problems. Along with the development of deeper network, researchers found out that gradient disappearance occurs when the sigmoid function approaches to 0 in the saturation region. en, ReLU is introduced in AlexNet to address this problem. Moreover, the operations using ReLU are simpler than those of the sigmoid function. Later, many scholars made various improvements based on ReLU and sigmoid functions. For instance, Leaky ReLU [15] solves the problem that ReLU is not activated when x is less than 0, PReLU [16] adds a parameter to make ReLU more accurate when x is less than 0, and RReLU [17] learns parameters automatically via the neural network. Here, PReLU is selected as the activation function which is shown in Figure 1, and its equation is given as follows:

Fully Connected Layer.
After convolutional computation, the high-dimensional feature maps will be classified and predicted through a fully connected neural network. is layer is often used in many network architectures such as LeNet, AlexNet, and GoogLeNet. e equation is given as follows: where I c and O c represent the number of input and output channels. From equation (4), the number of parameters in the fully connected layer depends on the input dimensions. If dimension reduction is not performed, the number of input channels might be massive, and many parameters will be generated. According to Lin et al. [18], the fully connected layer is prone to overfitting which hampers the generalization ability of the overall network. erefore, the later CNN architectures usually replace fully connected layers with global average pooling.

Efficient Light Convolutional Neural Network (ELNet)
An efficient light convolutional neural network (ELNet) is proposed to make the network architecture suitable for resource-constrained hardware. A schematic view of the network is depicted in Figure 2, where the red block is a depthwise separable convolution, the black dash line block represents an inception module, and the brown block is a depthwise separable convolution combining with atrous convolution. e details of the architecture are described in Table 1.
In Table 1, Conv dw represents a depthwise separable convolution, and d means stride in the atrous convolution.
e three convolutional modules used in ELNet are described as follows.

Depthwise Separable Convolution.
Depthwise separable convolution separates the original convolution into two parts for the purpose of reducing operations as shown in Figure 3.
Compared with the conventional convolution method, one convolutional kernel will generate only one feature map according to its input dimensions. However, depthwise separable convolution performs multiple feature maps corresponding to each dimension, and then a 1×1 convolutional layer is used to combine all the feature maps into one output. Although there is no difference between the output of the depthwise separable convolution and conventional convolution, the parameters of the depthwise separable convolution using one 3 × 3 convolutional kernel are much less than those of the conventional convolution method. e calculations are listed as follows: conventional convolution : where I w , I h , and I c represent the width, height, and channel of the input, respectively, k w and k h are the width and height of the convolutional kernel, and k c is the number of convolutional kernels in the convolutional layer.

Atrous Convolution.
Atrous convolution [9], as shown in Figure 4, enlarges the FOV of filters by incorporating the larger context without growing parameters. e advantages of using atrous convolution are allowing the user to filter a larger context instead of using a bigger size of kernel and reducing the usage of pooling layers which brings less operation consumption and accuracy improvement; besides, using less parameters can also avoid an overfitting problem.

Inception Module.
Inception module uses various convolution kernels to extract features so that the feature maps are able to contain local features and global features. e schematic view of conventional convolutional layers and inception module are displayed in Figure 5 as comparison. Although both of the methods can map to the same size of FOV, local features in Figure 5(a) might be washed out at the end. On the contrary, the wash-out problem will not be   Mathematical Problems in Engineering considered when using the inception module ( Figure 5(b)); however, fusing multiple feature maps is another question. In general, concatenation (Concat) and addition (Add) are two common methods; the former can retain characteristics of each convolution output but produce high-dimensional problems; in contrast, the latter does not have dimensional problems, yet relatively might lose the independence of each output.

Results and Discussion
To deploy the systems on resource-constrained hardware for real-time data processing, large-scale datasets such as PASCAL VOC, ImageNet, and COCO are not considered. us, CIFAR-10 and CIFAR-100, two well-understood and widely used datasets, were provided to verify the performance of ELNet. e experimental results including parameters, FLOPs, and accuracy were compared, respectively, with the other state-of-the-art CNN architectures such as GoogLeNet [5], ResNet-50 [7], MobileNet [10], and All Convolutional Net (All-CNN-C) [19]. e hardware specifications and predefined parameters used in this study are listed in Tables 2 and 3.

CIFAR-10 Dataset.
e CIFAR-10 dataset includes 60,000 colour images with the size of 32 × 32 in a total of 10 classes. To fit into the proposed network, bilinear interpolation is used to resize the images into 224 × 224 which provides more features than using the padding method. Table 4 shows the results in which the parameters and MFLOPs required in larger CNN models such as Goo-gLeNet and ResNet-50 models are very large. In other words, these models need longer training time and higher operations. To make the model suitable for general hardware equipment, models with less operations and lower complexity are more favourable. erefore, the proposed model is also compared with MobileNet and All-CNN-C which are also called light models. According to the results, MobileNet uses less parameters and MFLOPs than others; yet, the accuracy is lower than that of ELNet. Even though All-CNN-C has the least parameter requirements, its MFLOPs are the highest which means the training time could be decreased by using better graphics processing units, but this increases the cost of hardware equipment. ELNet reaches a tradeoff between accuracy and parameters/ MFLOPs which is closer to the purpose of this study than that of other methods.

CIFAR-100 Dataset.
e CIFAR-100 dataset contains 100 classes which are more than in the CIFAR-10 dataset. erefore, the accuracy shown in Table 5 is obviously relatively lower than the accuracy of classifying the CIFAR-10 dataset; yet, the accuracy of ELNet is still the highest.
To evaluate the effectiveness of three convolutional modules used in ELNet, Tables 6 and 7 show the results of classifying the CIFAR-100 dataset. Table 6 shows that using atrous convolution can not only widen the FOV which increases the accuracy from 67% to 69% but also reach the same accuracy (69%) as using a bigger kernel size. Additionally, the inception module has the ability to extract    (Table 7), concatenation shows better accuracy than the other two methods; however, it requires more parameters and MFLOPs; thus, the addition method might be the better choice for implementing the network in a resource-constrained environment. Overall, the proposed ELNet showed better performance in comparison with either relatively larger CNN architectures (GoogLeNet and ResNet-50) or light CNN architectures (MobileNet and All-CNN-C). e accuracy of ELNet is acceptable if the environment of the deployed system is considered. Although the proposed ELNet reaches 92.3% and 69% in the CIFAR-10 and CIFAR-100 datasets, respectively, the accuracy can be improved by using more complex networks. e three convolution modules with depthwise separable convolution, atrous convolution, and inception modules can also be extended to these complex networks to lower the number of parameters and operations and preserve the accuracy of classification as well.

Conclusions
e contributions of this study listed in the following confirm that the ELNet can effectively reduce model complexity but maintain fine accuracy: (1) ELNet successfully combines three convolutional modules, depthwise separable convolution, atrous convolution, and inception module, for reducing the number of parameters and operations in the model (2) ELNet requires only 2.1 million training parameters and 2.57 mega FLOPs based on the input image size that is equal to 224 × 224 (3) e accuracy of ELNet reached 92.3% and 69% in CIFAR-10 and CIFAR-100 datasets, respectively erefore, the proposed ELNet can be applied on embedded systems for image classification applications. In addition, the architecture can integrate other methods such as parameter pruning, recursion, or other learning methodologies to optimize the network for further research.