With the rapid development of 5G technology, its high bandwidth, high reliability, low delay, and large connection characteristics have opened up a broader application field of IoT. Moreover, AIoT (Artificial Intelligence Internet of Things) has become the new development direction of IoT. Through deep learning of real-time data provided by the Internet of Things, AI can judge user habits more accurately, make devices behave in line with user expectations, and become more intelligent, thus improving product user experience. However, in the process, there is a lot of data interaction between the edge and the cloud. Given that the shared data contain a large amount of private information, preserving information security on the shared data is an important issue that cannot be neglected. In this paper, we combine deep learning with homomorphic encryption algorithm and design a deep learning network model based on secure multiparty computing (MPC). In the whole process, we realize that the cloud only owns the encryption samples of users, and users do not own any parameters or structural information related to the model. In the experimental part, we input the encrypted Mnist and Cifar-10 datasets into the model for testing, and the results show that the classification accuracy rate of the encrypted Mnist can reach 99.21%, which is very close to the result under plaintext. The classification accuracy rate of encrypted Cifar-10 can reach 91.35%, slightly lower than the test result in plaintext and better than the existing deep learning network model that can realize data privacy protection.
With the rapid development of 5G technology, 5G is leading the evolution of IoT standard [
Neural network technology was originated in the 1950s and 1960s as perceptron, with an input layer, an output layer, and a hidden layer [
Perceptron with multiple hidden layers.
Multilayer perceptrons can get rid of the constraints of early discrete transmission functions and use continuous functions such as Sigmoid or tanh to simulate the response of neurons to excitation. In terms of training algorithms, Werbos’s back-propagation (BP) algorithm is used. This is what we now call a neural network.
According to the report, a series of companies, including Google, Facebook, Baidu, and Alibaba, have also publicly announced that artificial intelligence will be their next strategic focus. Based on the technology backgrounds of these tech giants and their huge investments in deep learning, deep learning is now making extraordinary progress. Moreover, with the rapid development of cloud service mode, these enterprises deploy the corresponding deep learning model in the cloud as the computing vendors to provide services for users according to the different solutions. On the one hand, it provides users with powerful services. On the other hand, after uploading their data, such as photos, voice, and video, to the cloud computing, users will face the risk of data privacy disclosure because they cannot control their subsequent usage. In addition, we know that the neural network model will get better with the increase and diversification of the training dataset. However, in areas such as finance and health care, laws or regulations do not allow the sharing of personal data, so we may not be able to get a good deep learning model based on the limited data [
The appearance of homomorphic encryption makes the application of ciphertext in the field of deep learning possible. Rivest was the first to put forward the concept of homomorphic encryption in 1978 [
Although the theory of homomorphic encryption can be arbitrary calculation, but in practice, we should pay attention to the following characteristics of the encryption: only support integer data; the depth of multiplication needs to be fixed, so the addition and multiplication cannot be carried out indefinitely. And homomorphic encryption does not support operations such as comparison and maximization. Therefore, homomorphic encryption cannot be directly applied to deep learning.
In the following paragraphs, we will discuss these problems and then propose solutions. And our contributions are as follows: By combining deep learning network with the homomorphic encryption algorithm, we designed an architecture based on secure multiparty computing, that is, a series of distributed processing such as standardization and encryption of user privacy data at the edge, then reasoning based on cloud deep learning application, and finally returning the results to the edge for decryption. Thus, the privacy protection of user data is realized. In the whole process, we realized that the cloud only owns the user’s encryption sample, and the user does not own any parameters or structural information related to the model. We designed the corresponding CNN model and encrypted Mnist and Cifar-10, respectively, and then tested as the dataset. The results showed that the classification accuracy rate of the encrypted Minst dataset can reach 99.21%, which is very close to the test result under plaintext and also is close to the accuracy of state-of-art model. The classification accuracy rate of Cifar-10 encrypted dataset can reach 91.35%, slightly lower than the test result in plaintext and better than the existing deep learning network model that can realize user data privacy protection. And the unit sample takes an average of only 11 seconds to complete an inference in the cloud without parallel optimization.
The rest of the paper is organized as follows.
In Section 2, we mainly introduce the related work. In Section
There have been several excellent contributions in this field, such as follows.
Graepel et al. used a somewhat HE scheme to train two machine learning classifiers: linear mean and Fisher’s linear discriminate (FLD). They proposed division-free algorithms to adapt to limitations of HE algorithms. They focussed on simple classifiers such as the linear means classifier and did not consider more complex algorithms.
Bost et al. [
Dowlin et al. [
There are also a few recent work that look at privacy issues in training phase, specifically for the back-propagation algorithm. Bu et al. proposed a privacy-preserving back-propagation algorithm based on BGV encryption scheme on cloud. Their proposed algorithm offloads the expensive operations to the cloud and uses BGV to protect the privacy of the data during the learning process. Zhang et al. also proposed using BGV encryption scheme to support the secure computation of the high-order back-propagation algorithm efficiently for deep computation model training on cloud. In their approach, to avoid a multiplicative depth too big, after each iteration, the updated weights are sent to the parties to be decrypted and reencrypted. Thus, the communication complexity of the solution is very high.
As we know, privacy data processing is based on two considerations. One is to obtain the calculation result you want on the premise of not exposing your data to the other party. The other is that when the attacker breaches the protection, the obtained data or intermediate data are still meaningless ciphertext. Thus, we design the privacy of data using the neural network classification, and it can be described as follows: the two parties are Alice and Bob; Alice is data hold party; Bob is neural network holds a party. Alice want to be on the premise of not revealing information about yourself, to get what you want to use Bob’s neural network classification results. After Alice, to get the results, Bob does not know Alice’s data; Alice also does not know about Bob’s neural network.
Further description is as follows: first of all, we need to make it clear that there is already a trained model which is using plaintext in the cloud; our work is mainly focused on the stage of the reasoning model; according to the complexity of the dataset, we design different convolution neural network structures; network structure can learn more rich and more complex high-dimensional feature with better performance and better adaptability. In the reasoning stage, we present an interaction model based on secure multiparty computing, and the optimized Paillier encryption algorithm can be used to protect users’ privacy data and obtain the expected reasoning results.
In this section, firstly, we introduce the Paillier algorithm and how to optimize it to operate on real numbers. And then, we analyze the characteristics of the convolutional neural network and each layer of the network to combine it with the homomorphic encryption algorithm. Finally, the data interaction architecture based on MPC will be introduced.
In essence, homomorphic encryption refers to such an encryption function, which encrypts the plaintext by adding and multiplying operations on the ring and then encrypts the ciphertext after encryption, and the result is equivalent. Because of this good nature, people can entrust a third party to process the data without revealing the information. Below, we introduce the working steps and properties of the Paillier algorithm.
Take two large prime numbers
An integer
If
Among them,
If
The homomorphism of Paillier is shown as follows:
Paillier encryption is only defined for nonnegative integers less than Representing signed integers is relatively easy. We exploit the modular arithmetic properties of the Paillier scheme. We choose to represent only integers between [−max_int, max_int], where the max_int approximately equals We use Representing floating-point numbers as integers is a harder task. Here, we use a variant of fixed-precision arithmetic. In fixed precision, we can encode by multiplying every float by a large number (e.g., 1
Schematic diagram of numerical range.
Numerical expansion diagram.
The complete processing of real numbers scheme realization is available at
For Mnist data, we know that the size of every picture is 28
Convolutional neural network architecture.
Below, we will introduce layers in CNN, respectively:
Convolution kernels.
Average pooling and maximum pooling.
Schematic diagram of neuron structure.
(a) Sigmod and (b) ReLU activation function.
According to the above introduction of common levels in CNN, we find that the forward propagation of CNN mainly involves two kinds of operations: weighted sum and activation function. In the weighted sum, there are addition and scalar multiplication for the operations involved in the sample, which is very consistent with the property of Paillier which we introduced in the previous section. According to the description of homomorphism in the third section, we use code to encapsulate so that the ciphertext can be added directly, and scalar multiplication can be carried out, and the final decryption result is the same as the result obtained by plaintext participating in the same operation. Because the activation function is nonlinear, it does not meet the property of homomorphic encryption mentioned above. Next, we propose a solution to this problem.
In order to combine the Paillier algorithm with CNN, we must solve the problem that the nonlinear activation function cannot satisfy the addition of the homomorphic encryption algorithm and the property of scalar multiplication homomorphism.
Thus, we draw out secure multiparty computation (which hereinafter referred to as MPC). The roots of MPC lie in a work by Yao et al. [
Based on the above description and considering the actual application scenario, we designed our own interactive model. As shown in Figure
Data transfer architecture.
We define
As shown in Figure
Data transfer architecture.
In this section, we introduce our experiment. We implement our scheme using Python3.6 on a server with Intel(R) i7 CPU, 32G RAM, Nvidia GeForce GTX 2080Ti GPU.
First of all, in the CNN architecture based on Figure
CNN architecture.
In Table
Main parameters in model training stage.
Parameters | Value |
---|---|
Batchsize | 128 |
Learning rate | 0.001 |
Convolution kernel 1 | (3, 3, 20) |
Convolution kernel 2 | (3, 3, 50) |
Pooling layer filter | (2, 2) |
After 500 rounds of iteration, the test set can reach a high accuracy of 99.62%, as shown in Figure
Accuracy and loss curve during iteration: (a) train and test accuracy; (b) loss per iteration.
In the sample of Minist datasets, each pixel on the sample is processed with the improved Paillier homomorphic encryption algorithm, and the subsequent input model is used for reasoning. However, in order to facilitate analysis, here we need to ensure that the encrypted numerical is between [0 : 255] and only can be used to display the cipher image, and the Paillier algorithm generated in the ciphertext is generally very large, so we can perform modulo 256 operation on ciphertext data here to ensure that the gray value of each pixel of the image used for display is also between [0 : 255]. As shown in Figure
Sample, encrypted sample, and encrypted sample histogram.
After uploading data from the user to the cloud, the output of each layer of the model interacts with the local area, and the time for encryption and decryption has the greatest impact on the time efficiency. It is necessary to note because of the cloud to maintain a persistent connection with the local, so the attacker disguised as a user sends a false data to the cloud easily to get the result of the model parameters; so, from the reasoning process, we can show the final full connection layer with a certain probability to join false neurons or random disturb neurons [
It can be seen that the main factor determining the reasoning time of the whole model is the amount of data to be encrypted and decrypted as well as the number of times. According to our algorithm, the amount of data here refers to the final data obtained by each layer of the model, rather than the convolution kernel weight and bias term parameters of the model itself in the reasoning process. As shown in Figure
Different layers of data dimensions.
The number of decrypted is equal to the number of model layers.
Next, we discuss the optimization of encryption and decryption efficiency. In order to improve the encryption and decryption time, we use GMPY2 library to optimize the homomorphism algorithm code. GMPY2 is a C-coded Python extension module that supports multiple-precision arithmetic. GMPY2 is the successor to the original GMPY module. The GMPY module only supported the GMP multiple-precision library. GMPY2 adds support for the MPFR (correctly rounded real floating-point arithmetic) and MPC (correctly rounded complex floating-point arithmetic) libraries. GMPY2 also updates the API and naming conventions to be more consistent and support the additional functionality.
We compared the time required to encrypt a sample with or without GMPY2. Table
When key length = 3072, the effect of GMPY2 library on the encryption efficiency of a sample.
Lib | CPU (%) | RAM (M) | Time (s) |
---|---|---|---|
No GMPY2 | 10.61 | 166.3 | 233.365 |
GMPY2 | 10.61 | 166.4 | 24.271 |
In addition, the selection of key length also has a great influence on the algorithm execution time. For example, Table
The influence of key length on the efficiency of sample encryption and decryption.
Key length | 512 | 768 | 1024 | 2048 | 3072 |
1 sample | 0.281 | 0.589 | 1.163 | 7.707 | 24.399 |
64 sample(s) | 13.175 | 35.506 | 72.886 | 462.563 | 1510.263 |
At different key lengths, the encryption and decryption time of one sample.
We know that the current mainstream asymmetric encryption algorithm is mainly based on the difficulty of factorization of large prime numbers, and Paillier is no exception. Therefore, although the encryption and decryption time required increases rapidly with the increase in key length, the corresponding security coefficient also increases. In addition, the length of the key length in the Parillier also determines the size of the data to be processed. According to the above, the gray value of the sample will be processed to be between [0,1] before input to the model. Therefore, in the whole reasoning process of the model, the value size generated will not be too large, and for the consideration of safety, the following experiment sets the Key Length to 512. In the actual industrial production environment, the length of key length can be appropriately increased according to the requirements of security.
In Table
Data dimensions, parameters, and decryption time in different layers.
Layer (type) | Output shape | Param | Decrypt time (seconds) |
---|---|---|---|
Input | (28, 28, 1) | 0 | N/A |
Conv layer | (26, 26, 20) | 200 | 2.073 |
Activation layer | (26, 26, 20) | 0 | N/A |
Average pooling layer | (25, 25, 20) | 0 | 2.511 |
Conv layer | (23, 23, 50) | 9050 | 5.291 |
Activation layer | (23, 23, 50 | 0 | N/A |
Average pooling layer | (11, 11, 50) | 0 | 1.211 |
Flatten layer | (6050) | 0 | N/A |
Dense layer | (256) | 1549056 | 0.055 |
Activation layer | (256) | 0 | N/A |
Dense layer | (10) | 2570 | 0.002 |
Activation layer | (10) | 0 | N/A |
Total params: 1,560,876; total time: 11.143 s.
Finally, we use the ciphertext in test sets to input model for classification prediction, and the result accuracy was 99.21%. As mentioned above, the model accuracy could reach 99.62% under plaintext. In order to verify the feasibility of the scheme on a more complex network, we also used Cifar-10 [
Comparison of model accuracy.
Model | Our | CryptoNetsc [ |
---|---|---|
Mnist accuray | 99.21% | 98.95% |
Cifar-10 accuray | 91.35% | Not provide |
Active function | Function | Square function |
In Table
Comparison of cloud single trip prediction time.
Our | Cryptonet | CryptoDL | |
---|---|---|---|
Single inference time | 11 s | 570 s | 320 s |
Activation function complexity |
In this paper, we combine deep learning with the homomorphic encryption algorithm and design a deep learning network model based on secure multiparty computing to ensure data privacy protection when users use the cloud deep learning model. In datasets and CNN model of varying complexity, we all got good results, which further verify the feasibility of deep learning as a service based on encrypted data. The classification accuracy rate of the encrypted two kind of dataset can reach 99.21% and 91.35%. This is a strong indication that our method can better ensure the security of users’ private data in AIoT. Next, we will try to use the encrypted data to directly train CNN and then find the optimization method.
The data used to support the study are included within the article.
A preliminary study of this work was presented in the conference of “2020 International Conference on Networking and Network Applications (NaNA).”
The authors declare that they have no conflicts of interest regarding the publication of this paper.
This research work was supported by the National Joint Funds of China (U20B2050), National Key R & D Program of China (2018YFB1201500), National Natural Science Foundation of China (62072368, 61773313, and 61702411), and Key Research and Development Program of Shaanxi Province (2020GY-039, 2021ZDLGY05-09, 2017ZDXMGY-098, and 2019TD-014).