probabil-ity image at location (dopt, eopt, fopt) is padded into the corresponding location of a empty volume that has the same size with the likelihood map. This volume serves as the output of template matching, which contains trivial non-organ structures.
In order to remove trivial objects, we firstly threshold this output volume and then smoothed its surface using an opening morphology operator. The generated results is the atlas-based segmented organ, which can be regarded as the initial segmentation results.
4.2.6 Refinement of Organ Segmentation
The final step of organ segmentation is to refine the initial segmentation result by using a Geodesic active contour (GAC) algorithm [10].
GAC is an active contour method which tries to solve a differential equation using an iterative method. The differential equation is the result of minimiza-tion of an energy funcminimiza-tion which moves a contour toward the true boundary of the object. For the GAC segmentation algorithm, an initial contour is required.
Its segmentation results are sensitive to the initial position. As explained in the previous section, after template matching, we can achieve an atlas-based segment-ed organ. Hence, in our work, the generatsegment-ed atlas-bassegment-ed segmentsegment-ed organ was used as an initial contour of the GAC algorithm to achieve a more accurate final segmentation result.
used for testing the performance. For all of the test CT dataset, phase-ART has 6 samples, phase-PV has 12 samples and phase-DL has 4 samples.
The algorithm was run using MS-Windows-based personal computer (Intel R CoreTM-i7 3770QM 3.40GHz and 16GB-DRAM). The programming environment was coded both in C++ language based on ITK [22] and MATLAB. Visualization of the shapes were performed using VTK [23].
For the assessment of our method, we evaluated the bone segmentation and registration results, the accuracy of organ bounding box, and compared our results to the other state-of-the-art segmentation algorithms.
4.3.1 Bone segmentation
All the CT data we used included different contrast-phases (the ART, phase-PV or phase-DL). For bone segmentation, the biggest difficulty was faced to various threshold range for different contrast phases. To investigate the robustness of bone segmentation for various contrast phases, we applied our bone segmentation method to 60 clinical CT volumes. The segmentation results of 6 typical cases (2 phase-ART, 2 phase-PV and 2 phase-DL) are shown in Fig. 4.11. The results show that performing the our method to segment the bones gives us accurate results for phase-ART and phase-DL. It confirms our bone segmentation method is robust to phase-ART and phase-DL. However, in some contrast phase-PV CT images, the intestines and colon (such as the blood vessels) may have overlapped intensities with the bones. Hence, some of our segmented bone contained trivial non-bone structures, as shown in the second row of Fig. 4.11.
As described in the previous section, probabilistic atlas-based organ segmen-tation poses a number of challenges. Accurate mapping of the probabilistic atlas onto the input CT volumes is difficult because of the variability of anatomical structures. In our proposed method, we used the bone as the landmark to only estimate a approximate region for the organ (the organ bounding box). Then the accurate registration of the atlas was performed by the use of template matching in the bounding box. The advantage of our proposed method is that the atlas is not registered onto the CT volumes directly based on the bones registration. Even though some errors exist in the segmented bone, it did not affect the whole organ segmentation result. The segmentation results of liver and spleen from phase-PV CT volumes were shown in Fig. 4.12. It can be seen that even the bone segmen-tation was not very accurate (the case of second row), we still can obtain accurate segmentation results for both liver and spleen. Our method was robust to bone segmentation.
Figure 4.11: The bone segmentation results with various contrast phases. The first row is the phase-ART images (2 cases); The second row is the phase-PV images (2 cases); The third row is the phase-DL images (2 cases).
Figure 4.12: Effect of the segmented bones on the liver/spleen segmentation results with the phase-PV CT images. The first row is the bone segmentation result only contains the bone parts; The second row is the bone segmentation result contains the bone parts and trivial non-bone parts.
4.3.2 Bone registration
Registration was an vital step in the whole process due to the large influence on the overall accuracy of the bounding box. Since the bone of the body is considered as a rigid part in the human body, we perform the affine registration to transform the bones.
The conventional atlas-based segmentation method only used a single reference.
Since the region of the CT volume varies greatly (some volumes will have a region from shoulder to leg, while some volumes will have a region with only the liver), it was difficult to register a bone to a reference accurately for the different regions.
Taking this into consideration, we prepared four different reference bones for bone registration. In order to validate the effectiveness and the feasibility of our four references, the comparison registration results of four typical bone data is shown in Fig. 4.13. The first row was the conventional method which only used one single reference bone (Reference 1). The second row was our method which used four different reference bones for bone registration. The gray areas correspond to the reference bone, the blue to the moving bone before registration and the red to the moving bone after registration. Fig. 4.13(b) demonstrates that the moving bone is inaccurately registered onto the reference bone by the conventional method (the first row) due to the large difference between the reference and moving bone, while it can be accurately registered by our method (the second row). It shows that improper selection of the reference results in incorrect estimation of the organ location. Various reference bones were necessary to accurately transform bones to get the desired effect.
Another factor, which may effect bone registration, is bone segmentation.
Fig. 4.12 shows two typical examples of bone registration in phase-PV. As we described in the previous section, some non-bone structures were included in the segmented bone in phase-PV. However, as shown in Fig. 4.12, the inclusion of s-mall non-bone structures had little effect on the bone registration due to the lower percentage of the non-bone structures among the segmented bone.
4.3.3 Accuracy of organ bounding box
The strong correlation between the organ bounding box and organ segmentation has been explained in [19]. The accuracy of bounding box played an immediate decisive role in the process of organ segmentation. To accurately segment the organ, we need to estimate the candidate region of one organ based on the con-structed bounding box. The concon-structed bounding box for each organ were trained separately and independently in our method. For the test data, using adaptive selection of appropriate reference bone, the transformation between the extracted bone of the test data and the reference bone can be computed by using affine
reg-Figure 4.13: Registration results of four typical data with different bone reference (Gray-reference bone, Blue-moving bone before registration, Green-moving bone after registration). The first row is the registration results with Reference 1 and the second row is the registration results with our four references ( Reference 1, 2, 3 and 4).
Figure 4.14: The estimated bounding box for the liver and spleen on Case 22.
istration. Then the organ bounding box can be transformed onto the test volume.
Thus, we can estimate the candidate region of organ for the test data.
As shown in Fig. 4.14 and Fig. 4.15, two typical cases were chosen to validate our method. They can be used for the objective evaluation of detection accuracy about the organ bounding box. The estimated bounding box can be observed from three directions: axial, coronal and sagital. In our work, the detected location was considered to be correct if most of the detected 3D rectangle contained the ground truth CT. As can be seen in Fig. 4.14 and Fig. 4.15, the organs (liver and spleen) were correctly located inside the estimated region in each of the CT cases. Three slices that pass through the detected position of the target organ are shown. The rectangle indicated the detected organ location (bounding rectangle of liver and spleen). These results show that the proposed approach can perform localization tasks successfully.
Figure 4.15: The estimated bounding box for the liver and spleen on Case 5.
4.3.4 Accuracy of liver and spleen segmentation
For the accuracy assessment of our method, we compared it with conventional and recently developed atlas-based segmentation algorithms by different metrics.
Dice Coefficient (Dice) The dice coefficient is one of the most popular methods to evaluate segmentation accuracy. This metric is given in percent and based on the voxels of two binary 3D volumes, with Vmanual as the manually and Vauto as the automatically segmented organ.
Dice= 2|Vmanual∩Vauto|
|Vmanual|+|Vauto| ×100% (4.4)
Tanimoto volume overlap (VO) The Tanimoto volume overlap between two sets of voxels Vmanual and Vauto is given in percent.
V O= |Vmanual∩Vauto|
|Vmanual∪Vauto| ×100% (4.5)
Root Mean Square error (RMSE) The root mean square error is given in millimeters and based on the surface points of two 3D images Vmanual and Vauto.
RM SE = v u u t1
N
N
X
i=1
[(Vmanual)i−(Vauto)i]2 (4.6)
where N represents the total number of surface points.
In order to make a comparison, we compared the results using the conventional method (ConSeg-ConAtlas), in which the transformation between the reference bone and the patient bone was directly applied to the conventional atlas, according to [4]. In addition, we compared our organ segmentation method with the iterative probabilistic atlas based on the Gaussian distribution analysis (ConSeg-IterAtlas) to remove irrelevant tissues [9].
A. Liver segmentation
To investigate the performance of our segmentation method, we randomly selected one real case in our dataset to be performed on liver and spleen segmentation.
Comparison the original CT image (Fig. 4.16(a)) with the corresponding likelihood map (Fig. 4.16(c)), revealed that the liver can be more easily distinguished from other tissues in the likelihood map. However, if we applied the atlas information on the likelihood map, the other tissues with similar intensity can be significantly reduced, as shown in Fig. 4.16(d)-(f). Wherein, a brighter voxel indicates a higher probability of being liver. The black region corresponds to the non-liver region.
Compared with Fig. 4.16(d)-(f), we can see that the false positives (other tissues with similar intensity) can be significantly reduced by the use of the conventional atlas in the Fig. 4.17. Moreover, it can be seen that there were still a lot of false positives in the conventional method despite using an iterative atlas, because we cannot align the iterative atlas to the patient volume accurately, unlike our proposed method that can achieve more accurate alignment. Template matching in our method was utilized to improve the accuracy of the registration.
The template matching was aimed to estimate an initial segmentation of the liver. If we threshold the output of template matching, it can give an initial segmentation of the liver. Furthermore, in order to obtain an accurate result, we refined the initial segmentation result using the GAC algorithm. The final segmentation result of the two cases is shown in Fig. 4.18. The red results are segmented liver slice, which are overlaid with the original CT slices. The results demonstrate that accurate segmentations has achieved.
Quantitative and comparative results of applying different methods to liver segmentation with different phases is presented in Fig. 4.19. The first 6 data corresponds to ART-phase cases (with the average Dice’s similarity coefficient
= 0.944), the next 12 data corresponds to PV-phase cases (with Dice = 0.929) and the remaining 4 data is phase-DL cases (with Dice = 0.912). It reveals that our proposed method was robust to the liver segmentation with different CT phas-es. Table 4.1 listed the average ofDice, V Oand RM SE between automated and manual segmentations with different segmentation methods for all test CT scans.
The results indicated that comparison of the conventional segmentation method
Figure 4.16: Comparison of different liver segmentation results on Case 11. (a) The original CT image; (b) The manual segmentation; (c) Likelihood map; (d) ConSeg-ConAtlas method; (e) ConSeg-IterAtlas method; (f) OurSeg-IterAtlas method.
Figure 4.17: The ROC of liver with different thresholds on Case 11.
Figure 4.18: The final liver segmentation result performed on two cases. The first row is Case 11 and the second row is Case 1.
Figure 4.19: Qualitative comparison of different segmentation methods for the liver. The first 6 data is ART-phase CT cases; the next 12 data is PV-phase CT cases; the remaining 4 data is DL-phase CT cases.
Table 4.1: Segmentation accuracy obtained by different methods.
Average ConSeg-ConAtlas ConSeg-IterAtlas OurSeg-IterAtlas
Dice (Liver) 0.771 0.820 0.930
VO (Liver) 0.633 0.699 0.870
RMSE (Liver) 3.886 3.685 2.906
Dice (Spleen) 0.606 0.616 0.922
VO (Spleen) 0.463 0.474 0.857
RMSE (Spleen) 3.104 3.080 1.992
Table 4.2: Evaluation of the MICCAI testing dataset.
Method VOE[%] RVD[%] ASD[mm] RSD[mm] MD[mm] Score OurSeg-IterAtlas 6.4 0.01 0.97 1.87 18.1 77.1
Peng et.al [24] 5.5 1.0 0.8 1.7 18.6 80.6
Lu et.al [25] N/A N/A N/A N/A N/A 79.9
Kainmiiller et.al [26] 7.0 -3.6 1.1 2.3 20.9 73
Lee et.al [27] 6.9 1.3 1.1 2.1 21.3 75
Linguraru et.al [5] 7.3 N/A 1.2 2.3 N/A 69
Heimann et al [28] 7.7 1.7 1.4 3.2 30.1 67
Saddi et al [29] 8.9 1.2 1.5 3.4 29.3 64
(ConSeg) with our proposed segmentation method (OurSeg) results indicate a sig-nificant improvement. The simulation verified that the performance of OurSeg (Dice >0.93) was much better than ConSeg for segmenting the liver.
Moreover, we compared our proposed method with the state-of-the-art segmen-tation methods from the MICCAI 2007 competition: Constrained convex varia-tional model (Peng et al. [24]), Iterative mesh transformation (Lu et al. [25]), Heuristic Intensity Model (Kainmiiller et al. [26]), Automatic Level-set method (Lee et al.[27]), Normalized probabilistic atlas (Linguraru et al. [5]), Statistical deformable model (Heimann et al.[28]) and Global-to-Local shape matching (Sad-di et al. [29]). We also validated our proposed method with the MICCAI training dataset. Our method was trained with our database which was described in the beginning of Section 3 and tested on the MICCAI test dataset. The quantita-tive validation utilizes five metrics [30] including two volume errors: Volumetric Overlap Error (VOE) and Relative Volume Difference (RVD) and three surface er-rors: Average Symmetric Surface Distance (ASD), Root Mean Square Symmetric Surface Distance (RSD) and Maximum Symmetric Surface Distance (MSD). The average errors for the five metrics are summarized in Table 4.2. Our algorithm achieved a score of 77.1 point for the MICCAI data using the validation tools pro-vided by the organizers of the MICCAI competition. The score of our method was observed to be slightly higher than the 75 point of average manual segmentation.
The results demonstrated that our proposed approach yields the high precision results with respect to liver segmentation. The accuracy of our automatic method was higher in comparison with the state-of-the-art automatic segmentation meth-ods such as Kainmller et al. (2007) 73 point, Lee et al. (2007) 75 point, Heimann et al. (2007) 67 point and slightly lower with interactive segmentation methods such as Peng et al. (2014) 80.6 point, Lee et al. (2015) 77.9 point. Fig. 4.20 shows the accuracy of our segmentation results with different number of training samples. It can be seen that the segmentation accuracy of our proposed method can be significantly improved by increasing the training samples.
Figure 4.20: The segmentation accuracy can be improved by increasing the number of training samples.
Figure 4.21: Comparison of different spleen segmentation results on Case 11.
(a) The original CT image; (b) The manual segmentation; (c) Likelihood map;
(d) ConSeg-ConAtlas method; (e) ConSeg-IterAtlas method; (f) OurSeg-IterAtlas method.
B. Spleen segmentation
The different spleen segmentation methods were compared in Fig. 4.21. It can be seen from Fig. 4.21(c), that the spleen can be more easily distinguished from other tissues in the likelihood map. The other tissues with similar intensity can be significantly reduced using conventional atlas (Fig. 4.21(d)).
Moreover, due to the spleen atlas not being aligned, it remnant a lot of other tissues in the conventional method despite using an iterative atlas (Fig. 4.21(e)).
As shown in Fig. 4.21(f), using our proposed method, the spleen’s location can be found more accurately based on the template matching. The accuracy of the estimated spleen probability is described previously by the characteristic curve analysis which is similar to the ROC analysis (Fig. 4.22). OurSeg-IterAtlas method
Figure 4.22: The ROC of spleen with different thresholds on Case 11.
Figure 4.23: The final spleen segmentation result performed on two cases. The first row is Case 11 and the second row is Case 1.
had significant differences for the ROC curve (with the threshold change) compared with ConSeg-ConAtlas and ConSeg-IterAtlas methods. Fig. 4.23 presents the final segmentation result for the spleen by the GAC algorithm.
Fig. 4.24 shows quantitative and comparative results from applying different methods to segment the spleen on 26 test cases. The first 6 data corresponds to ART-phase cases (the average Dice is 0.933), the data 7 to data 18 corresponds to PV-phase cases (Dice is 0.922) and the remaining 4 data are phase-DL cases (Diceis 0.906). Regarding the result of applying our method to different contrast-phases, we can conclude that our proposed method was robust for the spleen segmentation.Table 4.1 gives a clearer depiction of the corresponding accurate results. It lists that the average of Dice to be 0.922, V O is 0.857 and RM SE is 1.992 mm for our proposed method. It can be seen that OurSeg-IterAtlas method had better performance for segmenting the spleen.
Figure 4.24: Qualitative comparison of different segmentation methods for the spleen. The first 6 data is ART-phase CT cases; the next 12 data is PV-phase CT cases; the remaining 4 data is DL-phase CT cases.