In this paper, we aim to propose an image compression and reconstruction strategy under the compressed sensing (CS) framework to enable the green computation and communication for the Internet of Multimedia Things (IoMT). The core idea is to explore the statistics of image representations in the wavelet domain to aid the reconstruction method design. Specifically, the energy distribution of natural images in the wavelet domain is well characterized by an exponential decay model and then used in the two-step separate image reconstruction method, by which the row-wise (or column-wise) intermediates and column-wise (or row-wise) final results are reconstructed sequentially. Both the intermediates and the final results are constrained to conform with the statistical prior by using a weight matrix. Two recovery strategies with different levels of complexity, namely, the direct recovery with fixed weight matrix (DR-FM) and the iterative recovery with refined weight matrix (IR-RM), are designed to obtain different quality of recovery. Extensive simulations show that both DR-FM and IR-RM can achieve much better image reconstruction quality with much faster recovery speed than traditional methods.
As a means of connecting the proliferating embedded devices to the Internet, the Internet of Things (IoT) has drawn extensive attention in recent years. IoT has the great potential to significantly influence our lives and the way we interact with devices including sensors, security/surveillance cameras, mobile phones, home automation/control devices, and vehicle state meters [
The multimedia content, for example, image, audio, video, acquired from the physical environment possesses distinct characteristics as compared with the scalar data acquired by typical IoT devices, so the IoMT devices require much higher processing and memory resources. Moreover, the multimedia transmission is more bandwidth hungry as compared with the conventional scalar data traffic in IoT. Considering that the devices are usually low-computation capability devices and may run on battery with limited power, there is a great need for developing special multimedia data processing and transmission technologies that enable green computation and communications, that is, low-power and low-complexity signal processing, and energy efficient transmission. A core module of the multimedia data processing is compression, which aims to reduce the amount of data needed to be transmitted. Traditional compression techniques, such as the JPEG, JPEG 2000 algorithm for image compression, the MPEG, and H.26X algorithm for video compression, are generally too complex for low-cost IoMT devices. In this paper, we focus on the compressed sensing (CS) based image compression and reconstruction methods, aiming to design low-complexity and robust image transmission strategies for green IoMT applications.
The CS technique was first proposed by Candes et al. [
CS based image compression and reconstruction have already shown great potential for IoMP applications. However, there still remain two main problems. First, although the system complexity of “compression-reconstruction” has been greatly shifted from the encoder-end to the decoder-end, the encoding complexity is still too high to be handled by IoMT device when the size of the image is large. Second, although it is commonly assumed that the server node has abundant computation capability for the CS image reconstruction, there exist indeed situations where the decoder-end cannot afford to reconstruct the image, especially for real-time multimedia applications, for example, video surveillance using mobile handset to collect the IoMT packets. For the first problem, one solution is to divide the large image into small blocks and then compress each block sequentially. This method prominently reduces the computation complexity, but block processing destroys the boundary coherence between pixels, leading to obvious reconstruction quality degradation. To overcome this disadvantage, some new methods have been proposed, among which the separable image sensing encoder is the one that not only has affordable computation complexity, but also promises good reconstruction quality [
In our previous work, a separate-combine recovery has been proposed to yield a highly efficient recovery technique for CS image processing [
In this paper, we aim to propose a reconstruction method that can further reduce the computation complexity and, meanwhile, increase the reconstruction quality. To achieve this goal, statistical information is considered in the recovery method design. By analyzing the image representations in the wavelet domain, the energy distribution of natural images is found to be well fitted by an exponential decay model, which is then used as statistical a priori information in the recovery method. Based on the separate compression image sensing, the recovery process is also composed of two steps: the row-wise (or column-wise) intermediates recovery and the column-wise (or row-wise) final results recovery. The aim of reconstruction in each step is to obtain a solution conform with the statistical prior by introducing a weight matrix. Two recovery strategies are, respectively, designed in this paper: the Direct Recovery strategy with Fixed weight Matrix (DR-FM) and the Iterative Recovery strategy with Refined weight Matrix (IR-RM). The DR-FM strategy is aimed to achieve extremely high computation efficiency by employing the same weight matrixes for the two recovery steps, while, for the IR-RM strategy, the weight matrix for the second step will be iteratively refined, aiming to achieve more accurate recovery results. Extensive simulations and experiments have been conducted, as results show that the proposed recovery method with DR-FM strategy has a much better computational efficiency than traditional methods, while the recovery quality is better; the recovery with IR-RM strategy can achieve the best quality of recovery at the expense of degradation in computational efficiency slightly, yet still faster than the traditional methods.
The remainder of the paper is organized as follows. Section
In this section, we summarize the preliminaries and related work of this paper, including the CS basics, the image transmission in IoMT, and prior aided CS image reconstruction.
CS is applied for signal which is sparse or compressible. Under certain bases
In CS framework, a spare or a nearly sparse signal
The reconstruction of
If the measurement matrix
The original signal
In conclusion, the crux of CS theory is to reconstruct the sparse representation of the original signal by making good use of the sparsity nature. There are mainly two kinds of reconstruction algorithms.
They are a kind of recovery methods that can find out the locally optimal solution but not the globally optimal solution. Orthogonal matching pursuit (OMP) is a classical greedy method for CS recovery, and it is the base of many later proposed advanced methods, such as CoSaMP [
Different from the greedy algorithms, the convex optimization recovery methods aim to approach the globally optimal solution. The results by convex optimization recovery are usually more accurate, but the expense is more computation resources needed for the reconstruction. Convex optimization recovery can be used in situations where the recovery quality is preferred more than the recovery speed.
The IoT will enable connections of a wide variety of things, ranging from small sensors to the cloud of servers (data storage and analysis). With the explosive growth of IoT, the device numbers will increase by several orders of magnitude, resulting in the great requirement for mechanisms that can reduce the computation, power, and communication loads for the end-devices. For the IoMT devices, this requirement becomes especially stringent due to large amount of multimedia data to be processed and transmitted. In addition, for some application scenarios such as security video surveillance, both the device-end and the server-end need to be capable of handling real-time processing.
CS is one promising mechanism that can help to meet all the above requirements. First, CS is naturally a compression technique, so it can reduce the amount of data to be transferred, thus requiring less transmission power and bandwidth. Second, CS compression involves only linear operations, so the encoding complexity is extremely low compared with traditional nonlinear compression. For large image compression, by using methods such as separate sensing, the encoding complexity could further be reduced. Third, since the operation of CS compression is intrinsically global linear projection, every projected result has almost the same amount of information about the whole image. Therefore, CS compression is naturally robust to transmission loss. In other words, it can be regarded as a digital-fountain like erasure-correction codes. In all, the development of new and more efficient CS techniques for IoT is important, and our aim in this paper is to design reconstruction methods with not only low complexity but also high performance for real-time IoMT applications.
In the basic CS framework, only the signal sparsity or compressibility is explored by the reconstruction. In fact, physical signals including IoMT images often show more features other than sparsity or compressibility, such as the structural and statistical features, which can be used as priors to further improve the reconstruction quality or efficiency. For example, the signal wavelet coefficients can be naturally organized into a tree structure, and those significant coefficients that contribute to most of the image energy often cluster along a few branches of this wavelet tree. The condensing sort and select algorithm (CSSA) [
Besides structural prior exploration, statistical prior could also be used. In [
In this section, we will illustrate how the 2D images in IoMT applications can be compressed through the low-complexity separate sensing method and describe the existing corresponding reconstruction methods.
To reduce the computation complexity of compressed image sensing, the 2D separate compression has been proved to be an effective method [
Illustration of the separate sensing method for image compression.
Since the invention of separate compression, several corresponding recovery methods have been proposed in recent years, which will be described briefly in the next subsection.
Many existing methods of CS recovery are for 1D signals, not being able to be applied for 2D signals directly. A method of transforming the 2D image processing to 1D signal processing has been used for years by using the Kronecker product [
The original image signal
Let
Equation (
The 2D signal recovery has been transformed to a 1D signal recovery by (
According to the separate compression, the signals
The crux of 2D-OMP [
Separate recovery is a recovery method featured by its low computational complexity [
The recovery is composed of two steps: column-wise recovery and row-wise recovery. Let the intermediate
As is shown in Figure
Illustration of separate recovery.
Both of the two steps are to solve an underdetermined system of equations. The OMP algorithm can be chosen as the basic algorithm, by treating
This paper is aiming to propose a low-complexity yet high-performance recovery method for compressed image sensing. For the encoder-end, the separate sensing method described in Section
The image representation coefficient matrix in wavelet domain has significant characteristics, where low frequency components concentrate at top left corner and high frequency components concentrate at bottom right side. The matrix
The length of each energy level and the average absolute values of pixels contained in each energy level.
Level | Length of level | The average absolute values of pixels |
---|---|---|
1 | 8 | 180.4892 |
2 | 8 | 42.2611 |
3 | 16 | 27.1487 |
4 | 32 | 17.0842 |
5 | 64 | 9.800 |
6 | 128 | 5.0789 |
7 | 256 | 2.5370 |
|
|
|
Illustration of the wavelet coefficient blocks.
The ladder-like decaying attributes of wavelet representations for images.
As the energy level index goes up, the absolute values shows substantially decaying. And the exponential decaying can be used to fit it; as Figure
Curve fitting for the decaying of image wavelet representations.
To satisfy different application demands in practice, two recovery strategies are proposed in this paper, that is, the DR-FM strategy and the IR-RM strategy. In each strategy, the recovery process is composed of two steps, that is, the recovery for row-wise (or column-wise) intermediates and the recovery for column-wise (or row-wise) final results. Here, we take the column-wise first recovery followed by row-wise recovery as the example for description and analysis.
Based on separate recovery, reconstructing
The two steps have a similar equation form. Take the first step for example, that is, reconstruct
Based on the decaying model of the energy level, that is,
For simplicity, we can normalize the elements of the weight matrix
The result of (
Since this method involves simply matrix multiplication, we can directly obtain the solution of
Similarly, this method can be applied to the second-step reconstruction from
Finally, the sparse representation of the image can be obtained as
The whole process of this method is based on matrix multiplication. Therefore, this method has extremely low complexity and can improve the operation speed effectively.
By observing the differences between the recovery result of
The energy distribution weight matrix
Comparison between the estimation results and the weight matrix
Comparison between results of the first iteration and real values
Comparison between results of the first iteration and the weight matrix
Similar to DR-FM, the two steps of IR-RM can be expressed as
The first step of IR-RM is the same as DR-FM, and the recovery can be completed in a linear step
In the second-step recovery, let
Let
Similar to the DR-FM, the solution of the underdetermined equation system is
To obtain better recovery results, the weight matrix can be refined column by column, respectively, in the extra iteration of IR-RM as
The elements on the main diagonal satisfy
The reconstruction using the refined weight matrix can be formulated as
The solution of the above problem is
Since the results of the method with IR-RM derived from the second step are different from DR-FM’s, the solution with an extra iteration cannot be converted to the form of multiplication of matrixes and can only be solved vector by vector.
To demonstrate the efficacy of the proposed recovery strategy, simulation experiments have been performed with MATLAB in a 64-bit operating system with 8 GB memory and CPU clock speed of 3.30 GHz. Different sizes of Lena, Girl, and Man images are chosen in the experiments, respectively, and the results are evaluated by Peak Signal to Noise Ratio (PSNR) and reconstruction time.
The original image and recovery images of Lena, Girl, and Man are shown in Figures
The parameters and recovery PSNR of Lena, Girl, and Man.
Image size | Size after compression | Parameter |
PSNR of DR-FM | PSNR of IR-RM | |
---|---|---|---|---|---|
Lena |
|
|
|
29.0912 | 29.4311 |
Girl |
|
|
|
30.3810 | 30.5047 |
Man |
|
|
|
23.8872 | 24.1672 |
The simulation results of the image “Lena.”
The original image
Results by DR-FM
Results by IR-RM
The simulation results of the image “Girl.”
The original image
Results by DR-FM
Results by IR-RM
The simulation results of the image “Man.”
The original image
Results by DR-FM
Results by IR-RM
Based on the results of the three images, DR-FM and IR-RM are both effective for image CS recovery. Compared with DR-FM, the IR-RM recovery quality has been slightly improved after the extra iteration. As is described in Section
Simulation on Lena images using methods in the literature, including the 2D-OMP [
Recovery time comparison between different methods.
Recovery quality comparison between different methods.
Compared with existing methods in the literature, that is, the separate-combine recovery and the 2D-OMP, the two proposed strategies with statistic information aids (DR-FM and IR-RM) are improved on both computational efficiency (less reconstruction time) and recovering accuracy (higher PSNR). The improvement of computational efficiency is mainly because the proposed strategies involve only linear operation of products between matrixes or vectors, while the existing methods are intrinsically greedy algorithms involving multiple nonlinear iterations. The improvement of reconstruction quality is due to the use of statistical information, in terms of the weigh matrix setting which aims to reflect the energy distribution of natural images in the wavelet domain. The algorithm with the highest accuracy is IR-RM, while the method with the lowest computing complexity is DR-FM. As is noted above, the performance difference between DR-FM and IR-RM is due to the weight matrix refinement by the extra iterations in the second step. In all, the two strategies proposed in this paper are more promising for IoMT applications, and they can be applied flexibly according to the specific requirements, namely, to achieve better recovering accuracy or to be more computation efficient.
The method proposed in this paper relies on the statistics a lot. Although different images contain different statistical information, favorable result could be stably obtained as long as the fitting shows a fast decay. There is little influence on the final result, as the parameter
Recovery quality of Lena, Girl, and Man with different parameter
Therefore, the proposed method is not sensitive to the parameter
To propose low-complexity CS image reconstruction methods for the green computation and communications of IoMT, this paper proposed a method with two reconstruction strategies based on the statistical prior information. Simulations show that it can reduce the complexity greatly by making good use of the prior information of image representations in the wavelet domain, even when the original image has been compressed heavily. The computation of the proposed DR-FM involves only linear matrix multiplications. The computation complexity of the proposed IR-RM is slightly heavier than DR-FM but still decrease a lot compared with traditional methods. The reconstruction quality of DR-FM is much better than traditional methods due to the use of statistical prior information, and the extra iteration of IR-RM further improves the reconstruction quality. The two strategies presented in this paper can be selected flexibly according to the practical requirements of different IoMT application situations, that is, higher reconstruction quality requirements or less reconstruction time requirements.
The authors declare that there are no conflicts of interest regarding the publication of this paper.
This research has been sponsored in part by the National Natural Science Foundation of China (Grant nos. 61371102, 91638204, 61001092, and 61201144) and the Shenzhen Fundamental Research Project (Grant no. JCYJ20160328163327348).