Automatic Registration and Error Detection of Multiple Slices Using Landmarks

Objectives. When analysing the 3D structure of tissue, serial sectioning and staining of the resulting slices is sometimes the preferred option. This leads to severe registration problems. In this paper, a method for automatic registration and error detection of slices using landmark needles has been developed. A cost function takes some parameters from the current state of the problem to be solved as input and gives a quality of the current solution as output. The cost function used in this paper, is based on a model of the slices and the landmark needles. The method has been used to register slices of prostates in order to create 3D computer models. Manual registration of the same prostates has been undertaken and compared with the results from the algorithm. Methods. Prostates from sixteen men who underwent radical prostatectomy were formalin fixed with landmark needles, sliced and the slices were computer reconstructed. The cost function takes rotation and translation for each prostate slice, as well as slope and offset for each landmark needle as input. The current quality of fit of the model, using the input parameters given, is returned. The function takes the built‐in instability of the model into account. The method uses a standard algorithm to optimize the prostate slice positions. To verify the result, s standard method in statistics was used. Results. The methods were evaluated for 16 prostates. When testing blindly, a physician could not determine whether the registration shown to him were created by the automated method described in this paper, or manually by an expert, except in one out of 16 cases. Visual inspection and analysis of the outlier confirmed that the input data had been deformed. The automatic detection of erroneous slices marked a few slices, including the outlier, as suspicious. Conclusions. The model based registration performs better than traditional simple slice‐wise registration. In the case of prostate slice registration, other aspects, such as the physical slicing method used, may be more important to the final result than the selection of registration method to use.

Objectives. When analysing the 3D structure of tissue, serial sectioning and staining of the resulting slices is sometimes the preferred option. This leads to severe registration problems. In this paper, a method for automatic registration and error detection of slices using landmark needles has been developed. A cost function takes some parameters from the current state of the problem to be solved as input and gives a quality of the current solution as output. The cost function used in this paper, is based on a model of the slices and the landmark needles. The method has been used to register slices of prostates in order to create 3D computer models. Manual registration of the same prostates has been undertaken and compared with the results from the algorithm. Methods. Prostates from sixteen men who underwent radical prostatectomy were formalin fixed with landmark needles, sliced and the slices were computer reconstructed. The cost function takes rotation and translation for each prostate slice, as well as slope and offset for each landmark needle as input. The current quality of fit of the model, using the input parameters given, is returned. The function takes the built-in instability of the model into account. The method uses a standard algorithm to optimize the prostate slice positions. To verify the result, s standard method in statistics was used. Results. The methods were evaluated for 16 prostates. When testing blindly, a physician could not determine whether the registration shown to him were created by the automated method described in this paper, or manually by an expert, except in one out of 16 cases. Visual inspection and analysis of the outlier confirmed that the input data had been de-

Introduction
Physical sectioning and 3D reconstruction of a specimen is a common method to create computer models. A well known example is the "Visible Human" data set available on the Internet, which can be used for many 3D modelling purposes. Another example, is the reconstruction of prostates in order to use computer simulation to learn more about prostate cancer distributions [1].
Physical sectioning usually implies that the relative positioning between the slices will be lost. Several methods have been developed in order to automatically find the correct translations and rotations, usually depending on the outlines of the slices. Most commonly, automatic positioning (registration) is done by positioning the slices pairwise. Several different approaches exists, among others scale-space based search [2] and least square minimisation of a set of points [3,5]. These methods assume that the contour is known, for example extracted using thresholding, if applicable. The methods rely on a good starting point. The usefulness might depend heavily on the application. Stacks of slices are commonly improperly registered too tightly when using such methods. That is, the final 3D volume will be more straight and "cigar shaped" than the original shape of the specimen.
In some cases it is possible to make landmarks. That is, before the specimen is sliced, some information is stored into the specimen that can be localised in each slice. A simple way is to penetrate the specimen per-pendicular to the slicing direction with an ink filled needle.
However, ink tends to spread in the tissue and gives a diffuse landmark that is not reliable. Our group has developed a method to make landmarks by inserting paralel, sharp needles in the unfixed specimen [4]. The needles remain in the specimen during the fixation procedure and small holes can subsequently be identified in the sections. It is important to understand, that the landmarks will not be ideal -they are sometimes deformed because the slices may have been stretched implying no perfect match if several landmarks are used. Landmarks may vanish during preparation of the slices. The specimen itself can be deformed when the landmarks are positioned. A single landmark is enough to determine the translation of an object but two landmarks are needed to also find the rotation. Even two landmarks are seldom enough, since you can only detect deformation in one dimension using two landmarks. Thus, landmarks do not solve all problems, and new problems arise due to the landmarks.
Automatic positioning is done by small movements and rotations of the slices in directions that improve the fit, that is, make the sum of distances between the landmarks smaller. The sum of distances is called the "cost function". The current rotations and translations are used as input and the output of the function is the sum of distances (or some other measure of the quality fo fit). Commonly, the slices to be registered are assumed to be similar [6]. Unfortunately, when slicing prostates ( Fig. 1) at the interslice distance of 2.5 or 5 mm as are the assumptions in this report, the shape can vary significantly between any two slices. In this report, we present a method that registers all slices simultaneously using landmark needles that do not depend on the shape of the specimen.

Methods
Prostates from sixteen men that underwent radical prostatectomy were formalin fixed with landmark needles [4] and sliced at 2.5 mm. This is the smallest possible slices to make when considering both available equipment and time consumption. Thinner slices cannot be created without step-sectioning the slices, which is a very time consuming process. The slices were computer reconstructed [1]. Figure 2 shows the position of the needles in the prostate.
The registration (positioning) of the slices in the computer was done automatically but also manually in order to be able to evaluate the computer method. The manual registrations was done using a computer software. Movement, scaling and rotation of one of two displayed super imposed neighbouring slices was made. Also shown graphically was the current x-and y-transformations of all slices. The process is fully described in [7]. The computer registrations were made with 5 different combinations of cost functions.
For the automated computer registration an algorithm that could search for the optimal position was needed. The Powell [8] search algorithm is widely used and is simple and straighforward. It iterates towards the solution by moving in the current steepest gradient direction in the multi-dimensional parameter space using some additional gradient conditions in order to avoid getting stuck with slow convergence in multidimensional narrow valleys. We applied that algorithm in a straightforward way in this project.
Two cost functions have been used. The straight forward cost function gives at output the sum of pairwise Euclidean distances between the corresponding landmarks. The second cost function is a model of the nee- Fig. 2. The rigid needles are inserted prior to fixation, but are not bound to each other. After the needles have been removed, the prostate is cut into slices, and each slice will contain four round traces from the needles. dles inserted in the biopsy. This cost function has as its input the current translations and rotations for all slices in the prostate and also all current rotations and translations for the four landmark needles. The output is a measure of how well the model fits the actual case. The observed behaviour for landmark needles in prostates is that slopes between pairs of needles are opposite in direction. This is taken care of in the model by giving a less good fit if the slopes are in the same direction. The optimisation is iterative. That is, for the current input parameters the quality, based on the cost function, is calculated and the parameters are updated in order to best improve the quality in the next iteration. The iteration stops when the quality improvement is below a certain threshold value. In short, the proposed cost function can be described as: The penalty, if the needles are numbered 1, 2, 3 and 4 in clockwise order, is calculated in pairs for each of the x-and y-projection of the needles angle in respect to the horizontal plane. The x-projection angle for each of needle 1 and 4 is compared to angle 2 and 3, respectively. For the y-projection, needle angle for each of needle 1 and 2 is compared to angle 3 and 4, respectively. The penalty for each of these pairs is (angle A + angle B ) n where n is typically 3.0. This may be interpreted as if the angles are the same size with opposite sign, the penalty will be low.
The following piece of pseudo-code shows the cost function (V is the vector containing the current values for all parameters, set by the optimisation algorithm): Foreach landmark Transform the landmark position using the transformation parameters given in V for the current slice Compute the corresponding needle coordinate using the parameters for the needle given in V Add to distance the distance between the landmark and the needle coordinate. Foreach pair of needles Add to distance a penalty based on the similarity of the angles, based on the pairwise difference between slopes. return distance To determine whether slices have been extraordinarily stretched or deformed, the resulting landmark positions have been used as input in a regression analysis. The inverse I of the correlation matrix for the Studentized residuals e * was calculated [9][10][11]. Using the matrix I, the mahalanobis distance to origin was calculated. If the distance was large (>χ 2 0.05 ) the point was considered as suspicious and indicated that the corresponding slice should be checked manually. Figure 3 shows the results for the different methods for four of the sixteen cases as the centre points for all landmarks superimposed in one image. In the registration using the landmark needle model, the points line up not in random order but with a very good fit of the model. To compare the fit, the cost function's output after optimisation is presented in Table 1. In order to be able to compare the manual registrations with the other registration types, the needle position for needles perpendicular to the slices were generated using the same cost function as in the model based registration after the manual registration had been applied to the set of slices. All optimisations were made using the Powell [8] optimisation algorithm.

Results
A blind test to compare the model registration quality compared to the manual registration was made. For each case, the physician had to answer if registration A was better, equal or worse than registration B, where A and B were the manual and model based registrations shown in random order on the screen. Before comparison, the slices in the automatic registration without landmarks were manually registered by a non physician in order to get a complete set of registered slices.
The physician who had made the manual registration went through all cases twice. In the first pass, the registration software used to register slices manually [1] was used to show the position of the slices in both the manual registration and the model based registration in parallel. In the second pass, the same procedure was used, but the volume rendering software [1] was used for visualisation. In both passes, the cases were shown in random order. In the first pass, two cases were marked as noticeably different. All cases were referred to as "equal" in quality. In the second pass, one case was marked as "worse", all other cases were marked as "equal" in quality. The case marked "worse" was the model based registration. This case was shown to have a transformed set of landmarks (Fig. 3, rightmost case) Fig. 3. The landmarks of four prostates were compared using different optimisation methods compared to manual registration. The restricted model works the same as the full model, but the needles must be orthogonal to the slice planes. The images shows all landmarks as seen from the top of the stack of slices using parallel projection. The 2D line up of points in the full model is not a coincidence -the slices are stacked so that the 2D projection of the 3D line is well shown. The rightmost case is the one where the automatic registration was marked as "worse" than the manual registration. Two groups of slices can be seen. Investigation of this case reveals that this is because of the treatment in the fixation phase, where the lower and upper half of slices has been treated differently. 106.4 a All slice positions have been optimised at once, using the cost model described in this report. b Same optimisation as a , but the extra cost for needle displacment has been removed from the table. c The distances have been measured from a vertical needle placed in order to minimise cost using the Powell optimisation algorithm after the slices had been registered manually. d Same optimisation as a , but the needles were restricted to be perpendicular to the slice planes. e The mean landmark positions for all slices of the unregistered slices defines the centre. Slices has been registered one at a time. f The landmark positions for a single slice of the unregistered slices defines the centre. Slices has been registered one at a time. g Same as f but when optimising, the squared distance was used in the cost function. Here, the Euclidean distances are shown. and in the manual registration the world knowledge of prostate shapes was used to compensate for this.
The registration of seven slices was marked suspicious by the software. Only one of them was considered erroneous by the physician. That slice was the one in the transition in the case which also was marked as "worse" than the manual registration.

Discussion
Since many years, computer models of the prostate have been used in oncology for planning of radiotherapy of prostate cancer. These models are based on computer tomography scans or magnetic resonance images [12]. In prostate pathology, computerized three-dimensional models have been used by several groups during recent years [1,[13][14][15][16][17][18][19][20][21]. The aim of these studies has usually been to create a model for biopsy simulation. Prostate cancer is nowadays most often diagnosed by ultrasound-guided transrectal core biopsies. As many as 18 to 56% of prostate cancers are isoechoic and, hence, not seen by ultrasound [22][23][24]. Therefore, biopsies are usually taken according to a standardized biopsy protocol. Several protocols for systematic core biopsies have been described in the literature and their ability to detect cancer and to estimate the extent of cancer has been investigated in numerous studies [25][26][27]. For the evaluation of biopsy strategies, computer models have obvious advantages. A large number of possible biopsies can be taken from the same prostate without harm for the patient. These prostate models are based on histological tissue sections with cancer outlined with India ink. The sections are usually horizontal whole mount sections, i.e., slices through the whole prostate taken perpendicularly to the rectum. The segmented horizontal sections are stacked and the volume between them interpolated. A major problem in the reconstruction is the registration of the sections, i.e., to position the slices as close as possible to the original anatomical relations. Usually, this is done simply by comparing the sections pairwise as regards the shape of the prostate gland and the outline of the cancer areas. There is an obvious risk that this subjective method yields an erroneous alignment of slices. To avoid such misalignments, there is a need of landmarks that can be traced from one slice to another. Inserting foreign material in the tissue may cause signif-icant technical problems when processing the tissue. When the blocks are cut, the foreign material may tear the sections. Therefore, we developed a novel method to create landmarks [28]. Briefly, four parallel needles are inserted through the unfixed prostate. After formalin fixation, the needles are removed and small holes can easily be identified in the stained sections. Landmarks for alignment of the sections are particularly valuable in the sections close to the apex and the bladder neck where the prostate lacks natural landmarks such as the boundary between the transition zone and peripheral zone, the midline groove of the rectal surface and the ejaculatory ducts. The relations between different tumor foci in adjacent sections can be better analyzed when perfect alignment is obtained. The landmark needles do not consume tissue, but compress the surrounding prostate. It is essential to leave the needles in place until fixation is complete; otherwise the holes will collapse when the needles are removed.
The aim of this study was to develop an algorithm for automatic registration using the paralel landmarks. We were able to demonstrate that automatic registration of prostate slices is as good a manual registration as described in the last section. The automatic error detection correctly finds the erroneously registered slices. Slices that had been physically stretched prior to the computer reconstruction phase was the reason for misregistration. Only 16 cases were compared, which is too few to draw high precision conclusions. However, it is clear that automatic registration using the proposed algorithm is as good as manual registration if input data are considered to be undamaged. The main error in the resulting registrations are the deformations caused by the physical handling of the slices during the prostate fixation and slicing phases. Uniform slice thickness and minimal physical deformations of the slices are important factors for good results.
To use statistical tests with higher accuracy, more samples (i.e., slices) must exist. Unfortunately, the physical environment when slicing prostates does not provide this. In other registration problems where slices are used, the presented method will probably be even more useful.