Clinical validation of the use of prototype software for automatic cartilage segmentation to quantify knee cartilage in volunteers

Abstract

Background

The cartilage segmentation algorithms make it possible to accurately evaluate the morphology and degeneration of cartilage. There are some factors (location of cartilage subregions, hydrarthrosis and cartilage degeneration) that may influence the accuracy of segmentation. It is valuable to evaluate and compare the accuracy and clinical value of volume and mean T2* values generated directly from automatic knee cartilage segmentation with those from manually corrected results using prototype software.

Method

Thirty-two volunteers were recruited, all of whom underwent right knee magnetic resonance imaging examinations. Morphological images were obtained using a three-dimensional (3D) high-resolution Double-Echo in Steady-State (DESS) sequence, and biochemical images were obtained using a two-dimensional T2* mapping sequence. Cartilage score criteria ranged from 0 to 2 and were obtained using the Whole-Organ Magnetic Resonance Imaging Score (WORMS). The femoral, patellar, and tibial cartilages were automatically segmented and divided into subregions using the post-processing prototype software. Afterwards, all the subregions were carefully checked and manual corrections were done where needed. The dice coefficient correlations for each subregion by the automatic segmentation were calculated.

Results

Cartilage volume after applying the manual correction was significantly lower than automatic segmentation (P < 0.05). The percentages of the cartilage volume change for each subregion after manual correction were all smaller than 5%. In all the subregions, the mean T2* relaxation time within manual corrected subregions was significantly lower than in regions after automatic segmentation (P < 0.05). The average time for the automatic segmentation of the whole knee was around 6 min, while the average time for manual correction of the whole knee was around 27 min.

Conclusions

Automatic segmentation of cartilage volume has a high dice coefficient correlation and it can provide accurate quantitative information about cartilage efficiently without individual bias.

Advances in knowledge: Magnetic resonance imaging is the most promising method to detect structural changes in cartilage tissue. Unfortunately, due to the structure and morphology of the cartilages obtaining accurate segmentations can be problematic. There are some factors (location of cartilage subregions, hydrarthrosis and cartilage degeneration) that may influence segmentation accuracy. We therefore assessed the factors that influence segmentations error.

Backgroud

Biochemical cartilage information plays an even more important role than morphology in detecting early cartilage change. Developments in magnetic resonance imaging (MRI), such as three-dimensional (3D) quantitative MRI, allow for sensitive analysis of cartilage morphology. Quantitative parameters derived by MRI, such as T2* relaxation time, T2 relaxation time and T1rho can reflect biochemical changes in articular cartilage and can detect initial stages of cartilage degeneration [1,2,3,4]. According to some recent reports, T2* relaxation demonstrates a similar response in the assessment of articular cartilage and cartilage repair tissue [5,6,7]. Three-dimensional double-echo steady-state (3D-DESS) sequence is a common MRI sequence for morphological imaging of musculoskeletal diseases. Its reported sensitivity, specificity and accuracy for detection of cartilage lesions are 96.7, 75, and 93.7%, respectively [8]. It is usually used for the diagnosis of cartilage lesions [9]. Combining T2* mapping and 3D morphological imaging, therefore, has potential to enhance the assessment of articular cartilage.

Cartilage quantitative parameters, including cartilage volume and thickness, can be obtained noninvasively and can be derived automatically once a segmentation of a high-resolution 3D MRI sequences is available. These tools are valuable in a clinical setting because they can be used to quantitatively assess cartilage and save time in post-processing. Unfortunately, automated segmentation software can result in errors in segmentation of cartilage in some subregions [10]. Of the various cartilage segmentation algorithms, an approach proposed by J. Fripp et al. has been shown to be comparable or superior to other published automatic algorithms [11]. The deep learning algorithms would probably have high accuracy in cartilage segmentation, but it did not exceed individual network performance in cartilage thickness accuracy and voting ensembles [12]. However, there is no report on the relationship between the accuracy of deep learning method in cartilage segmentation and articular effusion and cartilage degeneration. This approach relies on a segmentation hierarchy, using machine learning to train three-dimensional active shape models to segment bone. Cartilage is segmented afterwards, by using a deformable model including the expected cartilage thickness and patient-specific tissue estimation. Recently, a study demonstrated that an approach that combines a deep convolutional neural network (CNN) and 3D simplex deformable modeling is useful for performing rapid and accurate cartilage and bone segmentation within the knee joint [13]. However, that segmentation algorithm relies on accurate recognition of the boundary between tissues, which is easily influenced by hydrarthrosis and by edge blur caused by cartilage degeneration. The segmentation accuracy of that method remains to be clinically validated.

There are some factors (location of cartilage subregions, hydrarthrosis and cartilage degeneration) that may influence the accuracy of segmentation. We therefore assessed the influence factors for the error of segmentations using the approach from Fripp et al. (2010), based on 3D DESS and T2* relaxation time data. Cartilage quantitative parameters, including cartilage volume and thickness, can be obtained noninvasively and derived automatically once a segmentation of a high resolution 3D MRI sequence is available. Accuracy of automatic cartilage segmentation was assessed by comparing results to those from manually corrected contours of knee cartilage. Moreover, mean T2* relaxation time of these cartilage subregions were also measured.

Methods

We examined 32 right knees of 32 volunteers, each of whom underwent MRI examinations. The volunteers included 13 males and 19 females, aged 21 to 37 years (mean 27.5 ± 5.2 years). Their body mass index (BMI) was between 17 and 28 kg/m² (mean 21.9 ± 2.5 kg/m²). This study was approved by the ethics committee of our hospital (2019–003-1), and all participants provided written informed consent. Inclusion criteria were: (1) age 18–40 years; (2) BMI < 28 kg/m²; (3) without knee infection, trauma, or surgery; and (4) without chronic diseases. Exclusion criteria were (1) knee injury; (2) morphological damage to articular cartilage; (3) knee pain or other positive symptoms; and (4) contraindication for MRI examination.

Scans were performed on a 3 T MR scanner (MAGNETOM Verio, Siemens Healthcare, Erlangen, Germany) using the 8-channel knee coil. The 3D knee images were obtained to show the high-resolution morphology using a 3D-DESS sequence with selective water excitation. The imaging parameters were: voxel size 0.63 × 0.63 × 0.68 mm³, TE 5.17 ms, TR 14.45 ms, flip angle 25°, matrix: 256 × 256 × 240, FOV: 160 × 160 mm². The sagittal T2* maps were obtained utilizing 5 echoes for the fit: TE = 4.36, 11.9, 19.44, 26.98, 34.52 ms, TR = 1340 ms, FOV = 160.0 × 160.0 mm², flip angle 60°, matrix: 384 × 384, slice thickness 3.0 mm.

A senior-level radiologist, who was blinded to the volunteers’ clinical information, evaluated the extent of cartilage degeneration and hydrarthrosis. Cartilage score criteria was obtained using the Whole-Organ Magnetic Resonance Imaging Score (WORMS) and ranged from 0 to 2. Knee cartilage was automatically segmented into 21 subregions [14] using post-processing prototype software (MR Chondral Health, version 2.1, Siemens Healthcare, Erlangen, Germany). This software automatically divides the knee cartilage into three main parts—femoral, patellar, and tibial cartilage—consisting of 21 cartilage subregions. The T2* maps were automatically registered to 3D DESS images by prototype software. The cartilage volume and mean T2* relaxation time for each subregion were also derived automatically by the software. The corrected slice was the slice that needs to be manually adjusted after automatic segmentation (Fig. 1). The T2* relaxation time of cartilage in the knee was measured by the same doctor twice a week apart to test consistency among observers. The automatic segmentations were manually corrected to increase overall segmentation accuracy. The Dice coefficient was used to quantify the amount of change performed on the automatic segmentation [5, 11]. The Dice correlation between automatic segmentation A (fA(x)) and manual correction based on the automatic segmentation B (fB(x)) was defined as in Equation: Dice (fA(x),fB(x)) = 2·fA(x)·fB(x)/(fA(x) + fB(x)). Levels of hydrarthrosis and cartilage scores (by WORMS) were determined to analyze their influence on the segmentation accuracy of each cartilage subregion.

Fig. 1

Cartilage segmentation: automated (A) vs automated plus manual correction automated (B): due to joint effusion, automatic segmentation identifies joint effusion as articular cartilage in the trochlea central and lateral of femur (black arrow)

Statistical analysis was performed with SPSS v.17.0 (IBM, Chicago, IL) and was expressed as mean ± standard deviation (Tables 1, 2). P values below 0.05 were considered to be statistically significant. Due to the small sample size, the paired rank sum test and independent sample t test were used to compare the regional differences of T2* relaxation parameter between the automatic segmentation and manual correction based on automatic segmentation in different groups by articular effusion and cartilage degeneration (Table 2).

Table 1 The cartilage mean volume ± standard deviation (SD), regional differences and Dice similarity coefficients between the automatic segmentation and manual correction based on automatic segmentation in different groups by articular effusion and cartilage degeneration

Table 2 The cartilage mean T2* value ± standard deviation (SD) and regional differences between the automatic segmentation and manual correction based on automatic segmentation in different groups by articular effusion and cartilage degeneration

Results

The intra-observer correlation coefficient was 0.99 for T2* measurement. The manual correction based on the automatic segmentation was commonly done in FMC, FMA, FTM, FTC, FTL, PLC, PMI, PMC, PMS, TLC, TLP, TMP, TMC, and TMA. Cartilage volume in the manual corrected group was less than in the automatic cartilage segmentation group (P < 0.05). Table 1 lists cartilage volume, P values, and Dice coefficient correlations for each subregion. In FMC, FMA, FTM, FTC, FTL, FLA, PLI, PLC, PMI, PMC, PMS, TLC, TMP, TMC and TMA subregions, the T2* relaxation value of manual corrected cartilage segmentation group was less than that of automatic cartilage segmentation group (P < 0.05). The Dice correlations between automatic segmentation and manual correction in different groups by articular effusion and cartilage degeneration are shown in Table 1. The mean Dice coefficient in FMP, FLP, FLC, TLP and TLA was close to 1, indicating an already high accuracy of the automatic segmentation prior to manual refinement. Cartilage T2* values and regional dissimilarities of all the subregions in the two groups are shown in Table 2.

With the increase in joint effusion, the Dice coefficient in the patella increased somewhat, but the difference was not statistically significant. The femoral condyle and patella had lower Dice coefficient (0.9969 and 0.9922, respectively) than the other regions of the knee cartilage when the cartilage score was 0 in the control group. The femoral condyle and patella had the lowest Dice coefficient (0.9900 and 0.9889, respectively) when the cartilage score was 2 in the hydrarthrosis group (Fig. 2).

Fig. 2

Cartilage score influence on Dice coefficient of the automatic cartilage segmentation software in the control and hydrarthrosis groups. Hydrarthrosis significantly decreased the Dice coefficient of moderate degenerated patellar cartilage and distal femoral cartilage but increased the Dice coefficient of mild degenerated proximal tibia cartilage. (WORMS 0 = normal; 1 = mild; 2 = moderate)

The average time for the automatic segmentation software to complete cartilage segmentation of a knee was around 6 min (for a processor model), while the average time for manual correction of a knee was around 27 min.

Discussion

These results suggest that use of the automated segmentation software results in a Dice coefficient of each subregion of higher than 0.9. The location of subregions, extent of hydrarthrosis, and level of cartilage degeneration are the most important factors affecting the accuracy of automatic segmentation. Automatic segmentation software can mistake some of the fluid accumulation at the edges of subregions, resulting in an overestimate of cartilage volume. These areas deserve greater attention during manual correction to increase segmentation accuracy in these parts. In all 21 subregions, the subregions with the most corrected slices were located in the medial anterior, central trochlea and lateral trochlea of the femoral condyle, medial inferior, and medial central of the patellar, and the medial anterior of the tibia condyle. These subregions were likely influenced by hydrarthrosis. Under the influence of hydrarthrosis, the Dice coefficient for automatic segmentation of the femoral condyle and patellar cartilage decreased when the cartilage score was 2.

The evaluation of T2* has been shown to be capable of characterizing different degrees of cartilage degeneration [15]. It had been proposed as a robust biomarker of articular cartilage degeneration in several joints [16, 17]. The advantages compared to T2 mapping include shorter scan times and higher SNR [5]. In this study, the presence of hydrarthrosis and a higher cartilage degeneration score decreased T2*. According to the literature, T2* may be susceptible to the spatial macromolecule architecture and its influence on water molecule mobility [5, 18]. In this study, failure of the automatic segmentation software to distinguish the contour of the cartilage occurred mainly in articular cartilage near the fluid accumulation. A segmentation algorithm with increased robustness against synovial fluid is currently being integrated, but was not available for testing at the time of this study. The boundary between articular effusion and articular cartilage was not clearly visible. The T2* relaxation times of cartilage subregions extracted with manually corrected segmentation was decreased compared to those extracted with automatic segmentation. Articular effusion was recognized as articular cartilage by the automatic segmentation, resulting in the increase of the T2* relaxation times in the uncorrected subregions.

There some limitations in this study. Although only normal volunteers were included, some undiagnosed cartilage degeneration was present. The accuracy of the segmentation of degenerated cartilage needs further evaluation. In addition, a larger sample size is required to increase reliability of results. A further limitation of this study was lack of inter-observer variability assessment for manually corrected faulty segmentations.

Conclusions

In general, automatic cartilage segmentation software had a high Dice coefficient and it can accurately evaluate the volume of cartilage. It provides quantitative information about cartilage morphology within an acceptable time range usually less than 10 min even on a laptop.

Manual correction can be used to improve the accuracy of the segmentation. The location of cartilage subregions and extent of hydrarthrosis and cartilage degeneration may influence segmentation accuracy. To derive exact results of T2* relaxation times of cartilage, manual correction of automatic segmentation is necessary, but even then using the software saves considerable time.