The most important finding in this study is that the accuracy of key variables in the NKLR was high, especially for primary reconstructions. Until 2020, data from the NKLR have been used in 71 publications in international journals [13]. Feedback to the surgeons and hospitals is presented in annual reports and at the annual conference of the Norwegian Arthroscopy Association. Findings from studies based on data from the NKLR have caused changes in clinical practice [13, 16, 19], and this study confirms that the underlying data leading to these changes are accurate. However, we did find some recurring errors of registration, where the most frequent errors had occurred in connection with revision surgery.
For most of the investigated variables there were no differences in registration accuracy between primary and revision surgery. The majority of the errors for the variables “activity at the time of injury” and “date of injury” were linked to forms completed after revision surgery. The surgeon often recorded identical information on “activity at the time of injury” and “date of injury” to what had been previously recorded for the primary reconstruction. The reason for this could be that the patient did not have any new major trauma leading up to the revision surgery, and that the surgeon therefore used information from the initial injury. A clear data definition guideline and automatic control of data entry in the digital data forms could potentially have decreased these errors. Due to the low accuracy for the variables “date of injury” (PPV 39%) and “activity at the time of injury” (PPV 52%) for revision surgeries, we recommend that the variables are not included in future research studies. We also recommend that “date of injury” (PPV 92%) in primary reconstructions is interpreted with caution, especially in analysis where this variable is thought to be of clinical importance.
Registration of cartilage lesions showed a high PPV of 100% and 91% for primary and revision surgery respectively. However, superficial cartilage lesions (ICRS grade 1) were found to be underreported to the NKLR compared with the patients’ medical records. In line with our findings, a similar study from the Danish Knee Ligament Reconstruction Register also found missing data on cartilage lesions due to imprecise registration [20]. Possible reasons for the missing data could be that superficial cartilage lesions are common and might not be considered an important finding, or that the lesions would be regarded as a “normal finding” given the age of the patient. We recommend emphasizing to surgeons the importance of registration of all cartilage lesions, as the incidence of superficial cartilage lesions might be higher than previously reported [21]. Several studies from the NKLR have included cartilage lesions in their analysis [22,23,24], although most of the studies only include deep cartilage lesions (ICRS grade 3 and 4), which in our study proved to have good validity.
Although the “diagnosis of ACL lesion” variable showed a PPV of 100% for both primary and revision surgery, we found that ACL injury was not always recorded on the form following revision surgery. This may have been because there is no guideline for this variable in the form, or simply that the injury was to the ACL graft, and not to the native ACL. Adding a simple guideline to this variable in the form could help prevent these errors in the future.
The “cause of revision” variable showed a satisfactory PPV of 94%. Nevertheless, we discovered that in 16% of the cases the value of the variable was missing in the NKLR. Failure of ACL reconstruction is often multifactorial [25, 26] and the direct cause can often be difficult to determine [27]. In the literature, the categorization of causes of revision is heterogeneous [25, 27,28,29,30,31]. With that in mind, a likely explanation for some of the “missing cases” might be that the surgeon was uncertain of the cause of revision and as a result refrained from filling in this variable altogether. The surgeon can manually report “other cause” of revision in addition to the already printed causes on the paper form. In 4% of the revisions, tunnel misplacement was manually reported as either the only cause of revision or in combination with another cause. Technical errors, especially tunnel misplacement, have previously been reported to be a significant cause of revision [32,33,34,35], and tunnel misplacement should therefore be added as an option to the list of causes.
Overall, this study demonstrates high validity of the data in the NKLR. We believe that important reasons for the high data quality are the thorough routines and highly qualified personnel of the Norwegian National Advisory Unit on Arthroplasty and Hip Fractures, with their considerable experience of managing national medical registries. The findings in our study suggest that the NKLR’s instructions to surgeons and data collection protocol are adequate. Routines are in place at the different hospitals to enable surgeons to fill in the form immediately after surgery without any inconvenience, in order to reduce the risk of recall bias. The time required to fill out the form seems to be acceptable, thus giving the register high compliance. The great majority of surgeons have filled out the forms [3, 10, 12], even though this was voluntary during the study period [13]. This suggests that the surgeons are genuinely interested in and motivated to contribute to the research and development of this field of orthopaedic surgery. The annual reports from the NKLR and the yearly update from the NKLR at the annual conferences of the Norwegian Orthopaedic Association and the Norwegian Arthroscopic Association are probably important in contributing to this motivation.
The NKLR is currently in the process of introducing a digital registration form that makes it possible to guide and give immediate feedback to surgeons filling in the form. This will further improve data quality as it allows for easily accessible descriptions of variable definitions, especially some of those found to have a low PPV in this study. Further, digital registration may eliminate missing data since data entry of key variables can be set as mandatory and it is possible to embed immediate automatic control of variables that have a logical association, for instance that the time of surgery comes after the time of injury.
The main strength of this study is its design, where five of the eight clinics included in the cluster were among the large-volume hospitals performing cruciate ligament reconstructions in Norway. Hospitals from the two most populated of Norway’s four administrative health regions were included, which are likely to be representative of the whole nation as the study includes both public and private hospitals. We have no reason to believe that systematic errors due to imprecise guidelines would vary between hospitals or regions. Furthermore, the study has a broad approach because it includes data from both primary and revision surgeries, which predominate in the NKLR database. The medical records were thoroughly evaluated by two reviewers, thus ensuring reference data quality. In addition, data were recorded in a predefined form, providing a low risk of information bias.
One weakness of using medical records as a reference standard is that the quality of the records depends on the particular surgeon, and thus information on some variables may be missing or incorrect. If the surgeon does not complete the record directly after surgery, there is also a risk of recall bias for intraoperative details.
Inclusion in the study depended on written consent forms from the patients. Unfortunately, only 89 of the 250 eligible patients (35.6%) returned the consent form. Even though we believe that data from 166 unique surgeries are sufficient to find any systematic errors, other errors might have been discovered if more surgeries had been included.
