Basic Unit Layer Rate Control Algorithm for H.264 Based on Human Visual System

In the process of the video coding, special attention should be paid to the subjective quality of the image. In the JVT-G012 algorithm for H.264, the influence of the human visual characteristic in basic unit layer rate control was not taken into account. This paper takes the influence of the human visual characteristic into the full consideration and offers ways to improve the subjective quality of the image. The visual characteristic factor, which is constituted by the motion feature and edge feature, is used to reasonably allocate the target bits, and then its quantization parameter is adjusted by encoded frame information. The experimental results show that, in comparison to the original algorithm, the proposed algorithm can not only control the bit rate more accurately but also make the peak signal to noise ratio (PSNR) stable, so as to improve the stationarity of the video image. The subjective quality of the reconstructed video is more satisfying.


Introduction
With the rapid development and popularization of the internet, electronic devices gradually become an indispensable part of our daily life, such as online broadcasting, online advertising, e-commerce, VOD, distance education, telemedicine, real-time video conference, smart phones, 3D video [1], VCD, DVD, HDTV, and streaming of multimedia video [2].However, the real-time data transmission and storage of multimedia data have become more difficult due to the limited communication bandwidth, especially in the video communication.Its high capacity of data has much difficulty in the process of transmission and storage.Thus with the limits of the bandwidth and the low storage capacity, the video coding aiming at using the least bits to represent image is very important.
The rate control plays an important role in the process of video coding.Since 70% of the information is obtained from eyes and the eyes is the final receiver of video information, it is vital important to take full advantage of the human visual characteristics to get a higher subjective quality of images.
According to the special structure of the human eyes, some scholars put forward some relevant algorithms [3][4][5][6][7][8][9][10].An adaptive bit allocation method is presented in [3], based on the space and time perception functions.The work in [4,5] presents a method based on the region of interest in the human eyes.A novel rate control algorithm is presented in [6], based on the visual perception characteristics.The work in [7] proposes a new digital video watermark method based on the human visual system (HVS).The work in [8] proposes a method based on the region of interest, aiming at distribution of the target bits.The work in [9] presents a video quality evaluation method based on the region of interest.The work in [10] presents an algorithm to distribute the target bits of the basic unit layer, which analyze the motion information and texture features.
Although the JVT-G012 is by now the most acceptable rate control algorithm, it still has shortcomings.The work in [11] proposes a joint rate-distortion optimization for the H.264 rate control algorithm with a novel distortion prediction equation, which avoids linear regression employed in other distortion predictors and can considerably speed up rate estimation.Multiple quantization parameters determination algorithm based on the statistics of the deviation measure is proposed in [12], which can achieve accurately QP.The work in [13] proposes a rate control technique for H.264/AVC using subjective quality of video.The work in [14] presents a complexity coefficient to combine the target bits.This paper presents a reformative basic unit layer rate control algorithm based on the HVS.The HVS was not taken into account in the JVT-G012 algorithm.Since eyes are the final receiver of the video information, so it is vital important to take the HVS into account in the video coding process.The HVS is very sensitive to the brim part and motion part.However each pixel of JVT-G012 algorithm is treated equally in the basic unit layer.Though the work in [10] takes advantage of the HVS, it is not comprehensive.In this paper the visual characteristic factor is used to improve the rate control in the basic unit layer.

JVT-G012 Algorithm
The JVT-G012 basic unit layer rate control mainly consists of three steps.First, it predicts the MAD of the current basic unit in the current frame.Then, it computes the target bit for the current basic unit.Last, it calculates the quantization parameter of the current basic unit and performs RDO.

Predict the MAD of the Current Basic Unit in the Current
where MAD cb is the predicted MAD of the current basic unit in the current frame.MAD pb is the actual MAD of the colocated basic unit in the previous frame. 1 and  2 are two coefficients of the predictive model, whose initial values are 1 and 0, respectively; after finishing encoding every basic unit, the coefficients  1 and  2 are updated.

Compute the Target Bit for the Current Basic Unit. The target bit for the current basic unit
where   and  unit are the number of remaining bits for the all uncoded basic units in the current frame and the number of uncoded basic units, respectively.MAD cb is the predicted MAD of the current basic unit in (1).

Compute the Quantization Parameter of the Current Basic
Unit and Perform RDO.
where  is the quantization parameter of the current basic unit.b is the target bit of the current basic unit in (2).MAD cb is the predicted MAD of current basic unit in (1). 1 and  2 are the first-order and the second-order model parameters of the quadratic rate-distortion model, respectively.They are updated in the process of encoding.

Improved Rate Control Algorithm
The reformative basic unit layer rate control algorithm mainly consists of two steps.First, it allocates the target bits based on the HVS.Then, it adjusts the quantization parameter and performs RDO.proposed algorithm assigns fewer bits to the unimportant region and assigns more bits to the region of interest in the process of allocating bits.It can achieve the goal of improving the overall video quality.This paper adjusts (2) with the visual characteristic factor (, ): where the motion vision characteristics and the edge vision characteristics are denoted by motion(, ) and edge(, ), respectively.

Motion Characteristics.
In the real scene, there exist two major motion scenes as following.
The whole scene changes a little while only parts of the objects move or change.At this time, human eyes are concerned much more about the moving and changing objects.That is, when MV avg is less than 4.5, When the whole scene moves fast, MV avg is more than 4.5, there are two subordinate situations in this case.
When there are many fast moving macroblocks, human eyes pay more attention to those objects that move little.This time MV is more than 2.5; motion(, ) can be expressed as motion (, ) = { { { 1 + (6.5 − MV) 10 2.5 < MV ≤ 6.5 3 MV > 6.5. ( When most of the objects move inconspicuously while only some of them move fast in the scene, human eyes pay more attention to the fast-moving part.This time, MV is less than 2.5; motion(, ) can be expressed as where the magnitude of motion vector for the th basic unit in the th frame and the magnitude of average motion vector in the remaining basic units of current frame are denoted by MV(, ) and MV avg , respectively.One has MV = √MV 2  + MV 2  , where MV  represents the magnitude of the macroblock motion vector in horizontal direction and MV  represents the magnitude of the macroblock motion vector in vertical direction.

Edge Characteristics.
This paper describes the edge characteristics of images with the variance because there is high variance in the edge area of images: where (, ) represents the variance for the th basic unit in the th frame.Medium() and max() represent the midvalue and the maximum value of the variance, respectively.

Adjust the Quantization Parameter and Perform RDO.
To consider the feedback information of the encoded frames, this paper adapts the quantization parameter adjustment coefficient.This paper uses  to adjust the quantization parameter, which is defined as the ratio of texture bits to the header bits: where    () is the average value of quantization parameters for all basic units in the previous frame.  is the quantitative parameter in the JVT-G012 algorithm.
After the adjustment, the algorithm took into account the encoded frame information.The proposed algorithm achieves a good rate control.The algorithm performs RDO and updates the model parameters.
Tables 3 and 4 show the comparison of the bit rate and the PSNR.The proposed algorithm gets much better PSNR than the Zheng algorithm for different video sequences and also controls accurately the bit rate.The proposed algorithm can improve the average PSNR for all test sequences, while the Zheng algorithm can improve PSNR for a part of test sequences.For example, Zheng algorithm gets less PSNR than the JVT-G012 algorithm for test sequences: mother-daughter, foreman, and hall.
Figures 1 and 2 show that the PSNR curve is flatter than the one obtained from the JVT-G012 algorithm.The proposed algorithm suppresses the sharp drop of the PSNR and improves the stability of the picture quality.
Figure 3 shows obviously that the highway keeps the edge part of images to which human eyes are sensitive, while the one obtained from the JVT-G012 algorithm distorts and influences the subjective feeling of eyes.
Figures 4, 5, and 6 are the comparison of the subjective quality.For the test sequences claire, mother-daughter, and silent, their facial and body parts have a dramatic decline  in image quality in the JVT-G012 algorithm.However, these features have better visual quality in the proposed algorithm.Figure 7 is the comparison of subjective quality for the test sequence carphone.It is found that the sensitive regions of eyes in the image, such as the face, clothing, and edge parts, have become blurred in the JVT-G012.However, these features have better visual quality in the proposed algorithm.

Conclusions
This paper proposes a reformative basic unit layer rate control algorithm by using the visual characteristic factor and the adjusted quantization parameter.The proposed algorithm allocates bits based on the HVS in basic unit layer and adjusts the quantization parameter with texture bits and the header bits of the current basic unit.The experimental results show that the proposed algorithm can control the bit rate more accurately and have a much better visual quality.Compared with the JVT-G012 algorithm, the PSNR can be improved by 0.2-0.7 dB.What is more, the PSNR has little fluctuation and the video image will become more stable.Compared with [10] algorithm, the PSNR for different test sequences has been improved obviously in this paper.The [10] algorithm has great effect on particular tests, while the proposed algorithm in this paper has universal applicability.In addition, the algorithm in this paper has great effect at low bit rates.

Figure 1 :Figure 2 :
Figure 1: PSNR curves of the highway sequence comparison for the JVT-G012 and the proposed algorithms.

3. 1 .Figure 3 :
Figure 3: Comparison of the subjective quality in the highway sequence for the JVT-G012 and the proposed algorithms.

Figure 4 :Figure 5 :
Figure 4: Comparison of the subjective quality in the claire sequence for the JVT-G012 and the proposed algorithms.

Figure 6 :
Figure 6: Comparison of the subjective quality in the silent sequence for the JVT-G012 and the proposed algorithms.

Figure 7 :
Figure 7: Comparison of the subjective quality in the carphone sequence for the JVT-G012 and the proposed algorithms.