Improved Crosstalk Reduction on Multiview 3D Display by Using BILS Algorithm

In multiview three-dimensional (3D) displays, crosstalk is one of the most annoying artefacts degrading the quality of the 3D image. In this paper, we present a system-introduced crosstalk measurement method and derive an improved crosstalk reduction method. The proposed measurement method is applied to measure the exact crosstalk 
among subpixels corresponding to different view images and the obtained results are very effective for crosstalk reduction method. Furthermore, an improved crosstalk reduction method is proposed to alleviate crosstalk by searching for the optimal integral intensity values of subpixels on the synthetic image. The derived algorithm based on modified Schnorr-Euchner strategy is implemented to seek the optimal solution to this box-constrained integer least squares (BILS) problem, such that the Euclidean distance between solution and its target decreases substantially. The method 
we develop is applicable to both multiview 3D parallax barrier displays and multiview 3D lenticular displays. Both simulation and experimental results indicate that the derived method is capable of improving 3D image quality more effectively than the existing method on multiview 3D displays.


Introduction
Crosstalk is a critical factor affecting the image quality in multiview three-dimensional (3D) displays [1], which is caused by the incomplete isolation of different image channels. In order to mitigate the Moiré fringe [2] and balance the horizontal versus vertical resolution [3] of 3D displays, the slanted parallax barrier or lenticular lens array is used [4]. However, the shape of subpixels on the display screen is rectangular or triangle and so on, and the shape of visible image observed through a slanted slit cannot exactly coincide with boundary of subpixels. This causes the viewers to observe not only the intended view image but also the other unintended view images and leads to the crosstalk observed at the optimal viewing positions on multiview 3D displays.
As introduced in [5], the available mathematical definitions of crosstalk are diverse and sometimes contradictory. In 2009, Huang et al. defined two terms [6]: system crosstalk that is independent of the content (determined only by the display) and viewer crosstalk that varies depending on the content. Most of literatures concerning the methodology [7,8] for measuring system-introduced crosstalk on multiview 3D displays provided the methods based on the illumination measurement. However, it is often difficult to measure the amount of crosstalk exactly, due to the complexity of the system, lack of measurements, reluctance of manufacturers to release data, and difficulty of making the measurement [5].
A number of methods [9,10] for crosstalk reduction on multiview 3D displays have been proposed in the literatures to date. For instance, lowing aperture ratio and fixing a special 3D barriers are both effective for crosstalk cancellation. But these methods are at the cost of the intensity or extra devices. Recently, Li et al. [11] have suggested a method to eliminate the crosstalk by correcting the subpixels values on synthetic image. Despite the fact that this method to some extent reduced the crosstalk, the crosstalk coefficients matrix was obtained by the illumination measurement. The experiment error of crosstalk measurement influenced the results of crosstalk reduction. Moreover, how to obtain the solutions to the equations of crosstalk elimination was not mentioned.
Few research efforts have been devoted to reducing crosstalk effectively on multiview 3D displays by taking it 2 Journal of Applied Mathematics as a box-constrained integer least squares (BILS) problem. Lattice reduction [12] is a powerful mathematical tool for solving diverse problems involving point lattices. In [13], the enumeration strategies by Schnorr-Euchner, Pohst, and Kannan were introduced. It was found that the Schnorr-Euchner strategy is substantially faster than the other two [14]. An efficient search method based on the Schnorr-Euchner enumeration strategy (see [15]) for solving integer least squares (ILS) problem was implemented in [16]. The method was then modified in [17] to solve the BILS problem by taking the box constraint into account.
In this paper, we extend the definition of systemintroduced crosstalk to the multiview 3D displays and present a mathematical method for exactly measuring the systemintroduced crosstalk observed at the optimal viewing position. Crosstalk between the neighboring view images is not always equal to the exact crosstalk between subpixels corresponding to different view images. Our method is applied to obtain the crosstalk coefficient matrix A, in which precise crosstalk between the subpixels is presented. The obtained results are more effective for crosstalk reduction method than for the one via the typical experiment. Moreover, we adopt the variable to simulate the slight movement of viewable image, while the viewer is moving parallel to the display screen in the viewing zone such that the more exact crosstalk at the optimal viewing position is obtained. One of major contributions in this paper is the proposal of reasonable measurement method of system-introduced crosstalk between the neighbouring subpixels above and below corresponding to the different view images.
In actual display devices, the subpixel values are constrained to the integer set Z. Without regard to it, the crosstalk elimination result still remains the biggist accumulated quantization error, which has the negative impact on image quality. That means the crosstalk elimination [11] on multiview 3D displays is not as complete as expected. Hence, it is effective by taking reduction process as a BILS problem. In this paper, we concentrate our efforts on developing a crosstalk reduction method, which gives a few modifications of Schnorr-Euchner strategy based algorithms given in [17], to correct for the synthetic image with the minimum quantization error. The experiments demonstrate that the proposed method reduces crosstalk effectively and greatly improves the 3D image quality. Moreover, based on the fact that the proposed method is based on the geometrical relationship of parallax barrier and subpixels, it is applied to eliminate crosstalk between the neighbouring subpixels above and below, rather than the neighbouring view images. If the distribution mode of view images changes on the synthetic images [18], that is, the neighbouring subpixels above and below do not correspond to the neighbouring view images, the method [11] becomes invalid. In contrast, the improved method still remains effective.
The following notation is used in the paper. R × and R denote the set of all real × matrices and the set of all real -vectors, respectively. Z × and Z denote the set of all integer × matrices and the set of all integer -vectors, respectively. I × ∈ Z × denotes × matrix with all entries equal to 1. For a real scalar , ⌊ ⌉ stands for its nearest integer, ⌊ ⌋ stands for the largest integer less than or equal to , and ⌈ ⌉ stands for the smallest integer greater than or equal to . The operation sign( ) returns −1 if ≤ 0 and 1 if > 0. For a matrix A, a denotes the th column of A, denotes the th entry of vector a , and A denotes the transposed matrix of A. Bold upper case letters and bold lower case letters denote matrices and vectors, respectively.

Crosstalk Measurement and
Crosstalk Reduction Method 2.1. Crosstalk Measurement Method. Multiview 3D displays provide 3D perception without requiring any form of special glasses or other user-mounted devices. Among them, parallax barrier displays and lenticular displays are prevalent in actual devices, as indicated in Figure 1. The synthetic image prepared for the 8-view 3D display is synthesized by 8 different view images, as can be seen in Figure 2. The rectangles represent the subpixels, and the numbers marked in them indicate view image numbers. The subpixels illuminated with color in Figure 2(a) are prepared for the observer at the first optimal viewing position. At the same time, the visible image observed through the slits is illustrated in Figure 2(b). They do not overlap perfectly; that is the main cause of crosstalk observed at optimal viewing position. We extend the definition of system-introduced crosstalk [6] from stereoscopic display to the multiview 3D display and precisely define the system-introduced crosstalk observed at the th optimal viewing position on the -view 3D display as follows: where ( = 1 ⋅ ⋅ ⋅ ) denotes the luminance measured at the th optimal viewing position with white in the th view image and black in all of other view images on the synthetic image and denotes the environmental light intensity. This definition is black-white crosstalk, since it uses full-black and full-white images in the testing scheme. Full-white and full-black are used because maximum leakage usually occurs when the pixels in the desired view image are full-black and the pixels in the other view images are full-white. The crosstalk between the neighboring view images on -view 3D display was considered in formula (1), rather than the crosstalk between the left eye image and the right eye image given in [6]. For the -view 3D display, ∑ =1, ̸ = − in formula (1) is replaced with 1 , which is the viewable area of unintended view images observed at the th optimum viewing position.
− is replaced with 2 , which is the viewable area of intended view image observed at the th optimum viewing position. Thus, we define the system-introduced crosstalk as follows:   where ( , ) denotes the subpixel that is in the th row on synthetic image observed through the th slit at the th optimal viewing position, V denotes the number of slits, and ℎ denotes the vertical resolution of the synthetic image. 3 denotes the viewable area of subpixel belonging to the intended view image and 4 and 5 denote the viewable areas of neighboring unintended subpixel above and below respectively. The distributions of 3 , 4 , and 5 are illustrated in Figure 3. The dotted lines present the edges of slits. The rectangles present the subpixels on the display screen.
Based on the image composition theory, the subpixels showing the intended view image are determined in all subpixels observed at the th optimal viewing position. Calculate 3 , 4 , and 5 and substitute the results into expression (2) to obtain the system-introduced crosstalk.
When Δ 1 ( ) ≥ − Δ 0 , the subpixel in the th row ⌊3 ⋅ tan ⌋ + 2th column on the synthetic image shows the color belonging to intended view image. Consider When Δ 1 ( ) < − Δ 0 , the subpixel in the th row ⌊3 ⋅ tan ⌋ + 1th column on the synthetic image shows the color belonging to intended view image.
(2) Crosstalk measurement when > arctan(1/3): when the subpixel is evenly divided by the left side of the slit, the intercept length on the bottom edge of this subpixel is Δ 0 = |(3 ⋅ tan − )/2|. When the subpixel in the th row is intercepted by the left edge of the first integral slit, When Δ 1 ( ) > Δ 0 , the subpixel in the th row ⌊3 ⋅ tan ⌋ + 1th column on the synthetic image shows the color belonging to intended view image. If If When Δ 1 ( ) ≤ Δ 0 , the subpixel in the th row ⌊3 ⋅tan ⌋th column on the synthetic image shows color belonging to intended view image.
Substituting these results into expression (3) obtains the system-introduced crosstalk for arbitrary multiview 3D display.
In the method given above, the top left corner of the visible image observed through the first integral slit and the top left corner of synthetic image coincide. Variable is used to denote the distance from the top left corner of the visible image observed through the first integral slit to the top left corner of synthetic image. Let be within the range of [0, ]. Then, Δ 1 ( ) = 3 ⋅ tan + − ⌊3 ⋅ tan + ⌋. The following calculation procedure is the same as that given above.
Varying from 0 to simulates the slight movement of visible image on the synthetic image, while the viewing position is moving parallel to the display screen in the viewing zone. The crosstalk varies with the variable . Thus, we define the system-introduced crosstalk as = min ∈[0, ] ( ) that is the system-introduced crosstalk observed at the optimal viewing point.

Improved Crosstalk Reduction Method.
For crosstalk reduction, the neighboring subpixels in the same column should be considered because the subpixels in same column on synthetic image show the same color, which is the main leakage among different view images. What we want to do is to correct the subpixel value on the synthetic image from ( , ) to ( , ) such that the leakage from the neighboring subpixels in same column will become useful for the image quality of multiview 3D displays; that is, where ( , ) and ( , ) denote the subpixel values in the th row column on the synthetic image before and after correction, respectively. That means that the intensity obtained at the th optimal viewing position is equal to the intensity of the th intended view image, after correction. To solve this problem, it can be written as follows: where B ∈ Z ×ℎ and X ∈ Z ×ℎ are the synthetic images before and after correction, respectively. A ∈ R ℎ×ℎ is the crosstalk coefficient matrix. The entry of A, ( , ) denotes the leakage degree from the subpixel in the th row th column to the one in the th row th column on the synthetic image. and are both the numbers from 1 to ℎ. We obtain ( , ) by the proposed method, which exactly reflects the crosstalk among the subpixels corresponding to different view images on the synthetic image, rather than among the whole neighboring view images.
In the actual display devices, ( , ) is set to be 0, if ( , ) < 0; ( , ) is set to be 255, if ( , ) > 255; and ( , ) is set to be ⌊ ( , )⌉, if ( , ) ∉ Z. This operation causes the fact that the unique solution to problem (12), X = A −1 B, may not be the optimal solution, which means crosstalk elimination does not perform completely.
If the rank of A is ℎ, which means A is full rank, the equation set has the unique solution X = A −1 B. However, B is constrained to the box B = {B ∈ Z ×ℎ : L ≤ B ≤ U}, where L = 0 × I ×ℎ and U = 255 × I ×ℎ . Hence X may be in following two boxes: We propose an alternative method to obtain the optimum solution to equations set (12). For the given B, X is the solution to a BILS problem: where ‖ ⋅ ‖ 2 denotes the Euclidean norm.
Given a real ℎ-vector b and a real ℎ×ℎ matrix A with full column rank, lattice reduction is applied to solve the problem: for = 1, 2, . . . , . The optimal solution to problem (14) is also the solution to (13). In lattice theory, A is called the generator matrix of the lattice L( ) = {Ax : x ∈ Z ℎ }, b is called the input vector, and (14) is referred to as a closestpoint problem, since it is the problem to find a point in the lattice which is closest to the given input point b .
At first, we transform the matrix A to an upper triangular matrix, which has good properties to make the search algorithm more efficient. Here, this can be performed by the QR decomposition of A: where P ∈ Z ℎ×ℎ is a permutation matrix, Q ∈ R ℎ×ℎ is orthogonal, and R ∈ R ℎ×ℎ is nonsingular upper triangular.
Then the proposed algorithm tries to solve (18). Note that, ifẑ is the solution to (18), thenx = Pẑ =ẑ is the solution to (14) andX is the solution to (13).
We provide a search algorithm which avoids some drawbacks of algorithm BGBF (Algorithm 1) [17]. Our algorithm is applied to obtain the correction result of synthetic image one column by one column. When we input b = b ( is the number from 1 to ), the correction result of this column z =ẑ is obtained. After times operations, synthetic image is processed to the correction resultẐ. Thus, we obtain the final correction resultX =Ẑ.

Results and Discussion
In this section, we aim to validate the improved crosstalk reduction method on multiview 3D displays, both qualitatively and quantitatively. We first give a specific example to demonstrate that the improved method alleviates the crosstalk on multiview 3D lenticular display effectively. Then computer simulation is carried out to compare the crosstalk correction results with typical method on two test sequences. Numerical results are presented to validate the performance of the improved method in terms of the mean square error (MSE) and peak signal-to-noise ratio (PSNR). Finally, a photometer is used for measurement to compare the crosstalk reduction results by the improved method and typical method. The experiment results indicate that the improved method eliminates the crosstalk more effectively and improves the quality of the 3D images further.
At first, we compare the crosstalk values obtained via the proposed method and measured through the experimental method. To verify the effectiveness of the proposed crosstalk measurement method, a photometer (FS-5500TLS) is used to measure the crosstalk at the optimal viewing position. At first, 8 black-and-white test images are obtained in which each view image is white (that means that the subpixel values corresponding to the certain view image are equal to 255) and, in turn, presented on the display screen. In the blackand-white test image, only subpixel values corresponding to the certain view image are equal to 255, and all of the other subpixel values are equal to 0. We perform the experiment, using a multiview 3D parallax barrier display screen (42HD, provided by Tianjin 3-D Imaging Technique Co. Ltd.) is used to display synthetic images. Then, measure environmental light intensity and each test images intensity, respectively, along the horizontal direction at the optimal viewing distance. The photometer is used for luminance measurements. The luminances and are obtained; then crosstalk = 0.38 is got through formula (1). Through the proposed crosstalk measurement method, we get the crosstalk = 0.38 when = arctan(2/9) on the multiview 3D display. The result well matches the actual measured crosstalk value. The close agreement between the proposed method and measured crosstalk results demonstrates that the proposed crosstalk measurement method can be used to measure the crosstalk on the multiview 3D display.
In our implementation,̂= arg min ( ) and the crosstalk coefficient matrices A ∈ R ℎ×ℎ are obtained by the proposed system-introduced crosstalk measurement method. The improved crosstalk reduction method is applied to calculate the correction values of subpixels one column by one column. However, when dimension ℎ is large, the search time becomes significant. So is rewritten as where ∈ R × . To determine , we analyze the relationship of neighboring subpixels in the same column. For example, when parallax barrier is slanted at the angle of arctan(2/9), we get the distribution mode of different view images' subpixels on the synthetic image, illustrated in Figure 2. Taking the first column on the left as an example, the third and fourth subpixels belong to the seventh view image and are observed at the seventh optimal viewing position. That means the third and fourth subpixels do not interact and are not related to each other. Consequently, we consider that every 3 subpixel

Input:
The upper triangular matrix R ∈ R ℎ×ℎ with positive diagonal entries, the vector b = b ∈ Z ℎ , the lower bound vector l = l ∈ Z ℎ , the upper bound vector u = u j ∈ Z ℎ , the initial hyper-ellipsoid bound . Output: The solutionẑ ∈ Z ℎ to the BILS Problem (18). values are correlative in one column; that is, = 3 when = arctan(2/9). Obviously, when < arctan(1/3), can be determined easily in the same way. When ≥ arctan(1/3), we can determine , according to the need.
First, we give a specific example to demonstrate the effectiveness of the proposed method for a group of images of a scene "Euro. " A multiview 3D lenticular display screen (46HD, provided by SuperD Co. Ltd.) is used to display synthetic images. The result before correction and the result by the method in [11] and our method are presented on the display screen, respectively. Figure 4 shows the photos taken at the fourth optimal viewing position. As can be seen the edges of "Euro" are blurring in the result before correction and it is still not good enough in the result by the method in [11]. Compared with Figures 4(a) and 4(b), the photo in Figure 4(c) is the clearest, especially at the boundaries of "golden bricks. " Furthermore, it can be observed from Figure 4 that objects' jagged edges (such as Euro' edges and the golden stick' edges) are still obvious in the result by the method in [11]. In contrast, our method well solves this problem; the phenomenon of contour jaggies largely disappears in the result by our method. It can be discovered from the experiment results that the proposed method reduces the crosstalk more effectively on the multiview 3D lenticular display. Then, we compare the correction results obtained by the proposed method and the method in [11] for two test sequences in terms of MSE and PSNR. Two test sequences are used in our implementation, namely, "Fireman" and "Lotus. " They are obtained from Tianjin 3D Imaging Technique Co. Ltd., Tianjin, China, which are 3D test sequences with proper parallax between adjacent views. The resolution of "Fireman" is 1024 × 600, while the resolution of "Lotus" is 1920 × 1024. To be fair, the commonly used metric MSE is used to evaluate the residual crosstalk, which is defined as MSE = (1/(ℎ × ))‖AX − B‖ 2 2 . Note that the MSE is converted to the PSNR based upon the relationship PSNR = 10log 10 (255 2 /MSE). Figure 5 plots the PSNR values versus both the frame and the view numbers in two sequences. For a closeup look, as an example, Figure 6 compares our correction results with the results by the method in [11]. The PSNR values versus the frame number for each view of "Fireman" are shown in the figure. Furthermore, the MSE curves for each view of "Lotus" are plotted in Figure 7, showing the frame-level residual crosstalk (MSE) of each view versus the frame number. It can be concluded that the improved method performs better than the typical method for both sequences in terms of MSE and PSNR.
The MSE per frame in results by the method in [11] and our method for two test sequences is compared. Figure 8 plots the MSE versus the frames for the sequences "Fireman" and  [11]. (c) PSNR of corrected "Fireman" by our method. (d) PSNR of corrected "Fireman" by the method in [11]. "Lotus. " As can be seen, the intensity observed at the optimal viewing position in the correction result by our method is very near to the intensity of intended view image; that is, the residual crosstalk reduces greatly in the results by our method.
We then estimate the average PSNR per view for the two test sequences. Table 1 lists the average residual crosstalk (PSNR) of each view of the two test sequences in results by the method in [11] and our method. These comparative results clearly demonstrate that the crosstalk reduction is better performed in our results.
To further verify the effectiveness of the proposed method, a photometer is used to measure the luminance at the optimal viewing position in our experiment. The screen we use for the measurements is the same 8-view 3D lenticular display as mentioned above. The screen is placed in a dark room and test images (before and after correction) and various reference images are displayed on it alternatively. The typical experiment measures crosstalk based on maximum pixel value. However, the pixel value is a random variable from 0 to 255 in the practical application, which is rarely 0 or 255. Furthermore, while the intended view image is observed, the other view images are all observed at the same time, rather than being observed alternatively. In typical experiment,  Before correction Method in [11] Our method when the luminance measured at the th optimal viewing position is near to the intended luminance in the th test image, the influence on the other viewing position is not evaluated. To be fair, we decide to use only one test image and 8 reference images for comparison. The test image is composed of 8 view images, 1 = 2 = 20, 3 = 4 = 220, 5 = 6 = 40, and 7 = 8 = 200, where denotes pixel value of the th view image. The luminance is measured at the th optimal viewing position by the photometer when the test image is presented on the screen. The intended view image's luminance ref for the th viewing position is obtained at the th viewing position when the th reference image is presented on the screen. 1 = 2 = ⋅ ⋅ ⋅ = 8 = in the th reference image; that is, there is no crosstalk for th view image. Absolute deviation is used to evaluate the performance, which is defined as | − ref |. The corrected test images by the method in [11] and our method are presented on the display screen, respectively. Table 2 lists the deviation values of the correction results for each view by our method and the method in [11]. These comparative results clearly demonstrate that luminance measured in our result is very close to the intended view image's luminance, which means the viewer crosstalk is eliminated largely by the proposed method.

Conclusions
Despite the fact that many researchers have been devoted to evaluating and eliminating the crosstalk in the stereoscopic display, there is still a lack of work on crosstalk reduction on multiview 3D displays. Typical methods for crosstalk measurement are based on the experiments, which cannot measure the precise crosstalk. In this paper, a systemintroduced crosstalk measurement method is proposed to obtain crosstalk when slant angle of parallax barrier is an independent variable. Through this method, the crosstalk coefficient matrix A is obtained. Moreover, we improve the lattice search algorithm to reduce the crosstalk on multiview 3D displays, which overcomes some shortcomings of algorithm BGBF. Both simulations and experimental results are presented to demonstrate that proposed method is capable of effectively reducing crosstalk on multiview 3D displays. In practical applications, our system-introduced crosstalk measurement method can be employed to study the performance of multiview 3D techniques and our crosstalk reduction technology can improve the multiview 3D viewing quality. We hope the reader can implement our algorithms without difficulty.