This paper proposes a robust realtime myocardial border tracking algorithm for echocardiography. Commonly, after an initial contour of LV border is traced at one or two frames from the entire cardiac cycle, LV contour tracking is performed over the remaining frames. Among a variety of tracking techniques, optical flow method is the most widely used for motion estimation of moving objects. However, when echocardiography data is heavily corrupted in some local regions, the errors bring the tracking point out of the endocardial border, resulting in distorted LV contours. This shape distortion often occurs in practice since the data acquisition is affected by ultrasound artifacts, dropouts, or shadowing phenomena of cardiac walls. The proposed method is designed to deal with this shape distortion problem by integrating local optical flow motion and global deformation into a variational framework. The proposed descent method controls the individual tracking points to follow the local motions of a specific speckle pattern, while their overall motions are confined to the global motion constraint being approximately an affine transform of the initial tracking points. Many real experiments show that the proposed method achieves better overall performance than conventional methods.
In company with the development of realtime threedimensional echocardiography (RT3DE), the demands for automated analysis methods of left ventricle (LV) assessment such as ejection fraction, motion analysis, and strain analysis are rapidly increasing. Nevertheless, most of the analysis methods are still based on the measurements in a few twodimensional (2D) slices, because they are available in clinical practice [
In the last decades, there have been numerous studies for tracking of LV wall motion such as the tracking methods using deformable models [
On the other hand, optical flow methods, which use the assumption that the intensity of a moving object is constant over time, provide the local motion information of myocardium. They are capable of measuring the LV volume as well as the myocardial wall motion analysis or strain analysis to detect LV abnormalities. After an initial contour of endocardial border is traced, each point on the contour tracks the specific intensity and speckle pattern in sequential images. However, it is problematic to track the endocardial border in ultrasound images with unclear speckle pattern or weak signals. In practical environment, there often exist some incorrectly tracked points due to ultrasound artifacts, dropouts, or shadowing phenomena of cardiac wall [
The estimation of endocardial border by the LucasKanade optical flow method. Case 1: a tracking point getting out from the real LV shape distorts the whole shape near the border with weak edges; (a) initially traced endocardial border and its tracking points at an ED frame, (b) the tracked result at the ES frame, (c) at the frame between ES and ED, and (d) at the next ED frame.
The estimation of endocardial border by the LucasKanade optical flow method. Case 2: the tracked points are irregularly spaced by indistinguishable speckle patterns; (a) initially traced endocardial border and its tracking points at an ED frame, (b) the tracked result at the ES frame, (c) at the frame between ES and ED, and (d) at the next ED frame.
In order to cope with these problems, we develop a new optical flow method equipped with a global motion constraint that is designed to prevent each tracking point from getting out of the endocardial border. In the proposed model, the LucasKanade (LK) optical flow method [
The proposed algorithm is capable of tracking LV border in realtime since its movement is directly computed from the difference between two sequential images via a simple matrix multiplication. For performance evaluation, we carry out various real experiments with Samsung Medison R&D Center (
Let
Horn and Schunk [
Lucas and Kanade [
As we mentioned in Section
Recently, Sühling et al. [
Compared with the approaches based on the LK method, Duan et al. [
Instead of maximizing the crosscorrelation coefficients, the velocity vector can be estimated by minimizing the sumofsquared difference (SSD) [
The block matching method uses similarity measures that are less sensitive to noise, of fast motion, and of potential occlusions and discontinuities [
The above three local methods have drawback in dealing with the problem of the contour shape distortion in the presence of locally weak signal corrupted by rib shadowing and other factors. Hence, we need to develop a method alleviating shape distortion.
The proposed method uses an affine transformation to describe a global motion that is synthesized by integrating local deformations. We denote the endocardial border traced at initially selected frame (e.g., endsystole or enddiastole frame) by a parametric contour
Here, we identify the contour
In our method,
The first term in (
The first term in (
The second term concerns a misfit between the estimated tracking points and their projection onto the space
To be precise, a careful computation yields
Hence, the second term in (
To compute the minimizer
The derivation of the EulerLagrange equation is given in the appendix.
For numerical algorithm, we replace the integral over
For notational simplicity, let the time
Then, the system (
This can be concisely written by
Therefore, we can directly compute the movement
For the parameter
For heuristic choice of parameter
We should note that if
From numerous experiments, we observed that
To investigate behavior of the parameter
Image frames by varying tissue/blood intensity ratio. We use echographic texture modeling and heart motion modeling to generate image frames with various contrasts.
Synthetic images: (a) original LV template, (b) speckle image with Rayleigh distribution, (c) with FisherTippet distribution, and (d) its smoothed image by a Gaussian filter.
For modeling of heart motion, we simulate a heart with the nonrigid motion integrating global and local deformations. Figure
The tracking points and displacements used in sequential synthetic images.
Tracking points






1  (152, 201)  (153, 196)  (154, 187)  (154, 183) 
2  (151, 176)  (152, 171)  (153, 165)  (154, 161) 
3  (155, 151)  (156, 147)  (157, 142)  (158, 139) 
4  (152, 128)  (153, 125)  (155, 120)  (156, 118) 
5  (142, 105)  (144, 103)  (146, 100)  (147, 98) 
6  (147, 82)  (149, 80)  (152, 78)  (153, 77) 
7  (167, 71)  (169, 70)  (171, 69)  (172, 68) 
8  (188, 84)  (189, 82)  (190, 80)  (191, 79) 
9  (207, 101)  (207, 99)  (207, 96)  (207, 94) 
10  (217, 124)  (217, 121)  (216, 116)  (216, 114) 
11  (224, 147)  (223, 143)  (222, 138)  (221, 135) 
12  (229, 172)  (228, 167)  (227, 160)  (226, 156) 
13  (229, 196)  (228, 191)  (226, 182)  (225, 178) 
Displacements





1 



2 



3 



4 



5 



6 



7 



8 



9 



10 



11 



12 



13 



Result showing the independency of
We first test the dependency of
Next, we test the dependency of
Graphs showing the relationship between
We test the proposed algorithm in clinical setting using many real data. We compare the performance of the proposed algorithm with some widely used tracking algorithms including the block matching tracking methods using sumofsquared difference (SSD) and crosscorrelation coefficient, and the LK optical flow. For experiments, we use the 35 cases of
A quantitative evaluation on the performance of the proposed tracking algorithm is done on real 2D image sequences. For computation of
For two representative cases among the 35 cases of 2D echocardiography data, the LV tracking results of the proposed method and the conventional methods are shown in Figures
Case I: real images with weak signals in endocardial border. The second and third rows are the results by regionbased tracking methods using sumofsquared difference (SSD) and crosscorrelation, respectively. The fourth row is the result by the LK optical flow and the final row is the result by the proposed method (
Case II: real images with indistinguishable speckle patterns in endocardial border. The second and third rows are the results by regionbased tracking methods using sumofsquared difference (SSD) and crosscorrelation, respectively. The fourth row is the result by the LK optical flow and the fifth row by the proposed method (
In Figure
For initial 10 sequential images, we compute
In Figure
Figure
Comparison results of LV contours using the Hausdorff distance
Case I
Case II
For performance evaluation of the proposed algorithm, we propose an additional assessment regarding the repeatability of local point along the forward and backward entire cardiac cycle. Let
Using this forwardbackward image
For the previous two representative cases, Cases I and II, Table
The comparison results of the proposed method with the conventional methods using FBTE, for Case I and Case II (in pixels).
Method  Case I  Case II 

Block matching (SSD)  3.6992  4.6566 
Block matching (crosscorrelation)  2.3396  5.5866 
Optical flow (LK)  2.6326  2.1521 
Proposed method  0.4052  0.7930 
Table
The comparison results of the tracking algorithms for the total experimental dataset of 35 cases. The errors are measured using the FBTE (in pixels).
Method  Mean of errors  Standard deviation of errors 

Block matching (SSD)  4.1936  2.4456 
Block matching (crosscorrelation)  4.4173  2.5684 
Optical flow (LK)  3.0685  1.2997 
Proposed method  0.6344  0.2884 
The proposed method controls the individual tracking points following optical flow by confining their overall motions by penalizing the misfit between the estimated tracking points and their projection onto the affine transform space
We have experimentally demonstrated that the proposed method is capable of performing robust realtime LV border tracking even in the presence of indistinguishable portions of the LV walls in echocardiography data. In practice, echocardiography data often contains edge dropout or indistinguishable speckle patterns in a local neighborhood of a tracking point which may bring the tracking point out of the endocardial border, resulting in distorted LV contours. The proposed method effectively deals with these problems by taking advantage of an LV shape space describing a global motion that is synthesized by integrating local deformations governed by the LK optical flow model. Various experiments show that the proposed method achieves better overall performance than the widely used conventional methods including the block matching tracking methods using sumofsquared difference (SSD) and crosscorrelation, and the LK optical flow.
The proposed method performs the LV border tracking by directly computing the displacements between two sequential images via a simple matrix multiplication. The computational time is affected by the size of the matrix, depending on the number of tracking points.
We also proposed a new performance evaluation method for LV tracking that is based on the forwardbackward tracking error estimation as shown in Section
The proposed technique can be extended to three dimensions by using 3D affine transformation as a global deformation.
In this appendix, we derive the EulerLagrange equation (
Using (
Simple arrangement of the above identities leads to the EulerLagrange equation:
Hence, it remains to prove the above identity. It suffices to prove
Then
This completes the first identity of (
The author would like to thank Samsung Medison R&D Center for supporting many ultrasound data and valuable suggestions. This work was supported by the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (no. 201281798 and no. 201181782).