This article has Open Peer Review reports available.
Cost-effectiveness of the SEN-concept: Specialized Emergency Nurses (SEN) treating ankle/foot injuries
© Derksen et al; licensee BioMed Central Ltd. 2007
Received: 31 March 2007
Accepted: 01 October 2007
Published: 01 October 2007
Emergency Departments (EDs) are confronted with progressive overcrowding. As a consequence, the workload for ED physicians increases and waiting times go up with the risk of unnecessary complications and patient dissatisfaction. To cope with these problems, Specialized Emergency Nurses (SENs), regular ED-nurses receiving a short, injury-specific course, were trained to assess and treat minor injuries according to a specific protocol.
An economic evaluation was conducted alongside a randomized controlled trial comparing House Officers (HOs) and SENs in their assessment of ankle and foot injuries. Cost prices were established for all parts of healthcare utilization involved. Total costs of health care utilization were computed per patient in both groups. Cost-effectiveness was investigated by comparing the difference in total cost between groups with the difference in sensitivity and specificity between groups in diagnosing fractures and severe sprains. Finally, cost-effectiveness ratios were calculated and presented on a cost-effectiveness plane.
No significant differences were seen between treatment groups for any of the health care resources assessed. However, the waiting times for both first assessment by a treatment officer and time spent waiting between hearing the diagnosis and final treatment were significantly longer in the HO group. There was no statistically significant difference in costs between groups. The total costs were € 186 (SD € 623) for patients in the SEN group and € 153 (SD € 529) for patients in the HO group. The difference in total costs was € 33 (95% CI: – € 84 to € 155). The incremental cost-effectiveness ratio was € 27 for a reduction of one missed diagnosis and € 18 for a reduction of one false negative.
Considering the benefits of the SEN-concept in terms of decreased workload for the ED physicians, increased patient satisfaction and decreased waiting times, SENs appear to be a useful solution to the problem of ED crowding.
Emergency Departments (EDs) worldwide are confronted with overcrowding due to physician shortage and a steady growth of patients visiting the ED [1, 2]. Long waiting times in the ED lead to unsatisfied patients. Moreover, crowding causes late diagnoses with the possibility of unnecessary complications. To cope with these problems, Advanced Nurse Practitioners (ANPs) and Physician Assistants (PAs) were introduced to take over certain diagnostic tasks from doctors to alleviate the physicians of some of the workload in the ED . These so-called midlevel practitioners have proven to be of good value in the everyday practice of the ED by treating certain injuries and conditions while maintaining a good standard of care and patient satisfaction [4–6].
However, there are also some disadvantages to the development: training is relatively long and expensive and trainees are withdrawn from the already tight nurses pool [7, 8]. To resolve these disadvantages and maintain the benefits of these practitioners, we conceived a new concept: the Specialized Emergency Nurse (SEN). Regular emergency nurses received an injury-specific course to assess and treat minor injuries themselves according to a protocol. SENs are trained to be flexible employees capable of treating common injuries alongside their regular nursing duties.
The first injury to test this method on was the ankle sprain, because it is a common injury for which validated clinical decision rules exist that could easily be implemented in an algorithm (Ottawa Ankle and Foot Rules) [9–13].
Before this study, the ability of SENs to clinically assess ankle and foot injuries was compared with junior emergency physicians (house officers [HOs]) in an interobserver agreement study . Furthermore, the accuracy of SENs in interpreting the accompanying radiographs was studied . The results of both studies were promising to such a degree that a randomized controlled trial, the SEN-trial, was set up to assess the ability and consequences of SENs diagnosing and treating patients with an ankle or foot injury compared with HOs. Outcome measures considered in the trial were accuracy of assessment, patient satisfaction, and waiting time. The clinical part of the study revealed comparable accuracy results between the SENs and HOs. The SEN group scored significantly better than the HO group on patient satisfaction and waiting times. These results are presented in a separate paper .
At present, no literature is available about the cost-effectiveness of deploying regular nurses to perform diagnostic and treatment tasks after a short, injury-specific course. ANPs and PAs, however, have been proven to be more expensive than HOs . The results of the economic evaluation that was performed alongside the SEN trial are presented in the current paper.
An economic evaluation was conducted alongside a prospective randomized controlled trial (SEN-trial) comparing House Officers (HOs) and Specialized Emergency Nurses (SENs) in their assessment of ankle and foot injuries in the ED. Assessment was performed by both groups according to a protocol based on the Ottawa Ankle and Foot rules. Effectiveness and cost-effectiveness were investigated. The protocol was approved by the scientific committee and medical ethics committee of the VU University Medical Center and was carried out in accordance with the Declaration of Helsinki (1989) of the World Medical Association.
Out of 32 certified emergency nurses, 16 volunteered to participate in this trial. The mean age of the nurses was 36 years (range, 26–56 years). The mean clinical experience in the ED for the nurses was 5 years (range, 6 months–12 years).
Before the study started, the nurses were trained in the anatomy and biomechanics (trauma mechanisms) of the ankle and foot, and were taught how to interpret the accompanying radiographs for the detection of acute fractures in a 2-day course, provided by a surgeon and a radiologist. Successful completion of the course led to the SEN qualification. Furthermore, all HOs, 24 in total, participated in the study. The HOs' mean age was 28 years (range, 26–30 years) and their mean clinical experience (in an ED) was 1 year (range, 6 months–1.5 years).
Patients were randomly assigned to 1 of the 2 treatment groups by computer, which allocated patients unstratified into the 2 observer groups in blocks of 20. All consecutive patients who came to the ED with an ankle or foot injury were invited to enter the study. Exclusion criteria were: age younger than 18 years or older than 65 years; trauma sustained more than 48 hours before presentation; mental or physical conditions known to complicate assessment of the injury; ankle/foot injuries as part of a more severe (poly)trauma; and prior injury to the ipsilateral ankle/foot that required surgery. Written informed consent was obtained from all patients. For more details on the trial, we refer to the clinical paper .
Clinical outcome measures
The included patients visited the outpatient clinic again after one week. One surgeon reassessed all patients to establish the definitive diagnosis (gold standard). The primary outcome measures were the accuracy parameters calculated as the sensitivity and specificity of both observer groups. These parameters were found by comparing the results of both observer groups to the gold standard (surgeon that reviewed the radiographs and reassessed ankle stability). The secondary outcome measure was patient satisfaction. The clinical outcome measures and their assessment are extensively described in the clinical paper .
Assessment of resource utilization
Data regarding costs directly related to the treatment of ankle and foot injuries were collected for each patient in the trial. The following data were taken into account: number of X-rays performed; use of crutches and duration of use; initial treatment (cast, pressure bandage or tubigrip); admittance to the hospital and duration of hospitalization; operation together with the used materials and duration of operation. Most cost parameters were collected at initial presentation at the ED. Few cost parameters (crutch use and hospitalization elsewhere) were collected at the control visit or by phone (for patients who did not return for the control visit). Data on crutch use were collected for the last 115 patients of the study. These data were used to estimate group means that were consequently imputed for the entire group. Other costs within the health care sector consisted of those associated with setting up the 2-day training course as well as providing it. Furthermore, the costs associated with the time SENs and HOs spent on patients in both groups were included. This was done by prospectively measuring the time spent on clinically assessing the patient as well as the time spent on interpreting the accompanying radiograph (if made). For practical reasons, these data were collected for a selection of patients and imputed for both groups.
Mean (SD) invested time spans per treatment group
Invested time per observer (SEN/HO)
Duration clinical assessment (min)
Duration radiograph interpretation (min)
Duration review radiograph (min)
Invested time per patient
Waiting time 1st assessment – radiograph (min)
Waiting time diagnosis – treatment (min)
Valuation of health care consumption; unit costs
Costs per unit health care resource used in the economic evaluation of SEN (year 2004)
Healthcare resource [Unit]
Cost per unit (€)
Crutch-rent per week
Lower extremity cast
Hospitalization per day
Operating room per hour
Osteosynthesis materials (mean)
Hourly fee SEN
Hourly fee HO
Hourly fee specialist
Waiting time patient
Training SEN (per SEN)
Statistical analysis was carried out according to the intention-to-treat principle. A total of 512 patients were included in the trial: 263 were randomized to the SEN group and 249 to the HO group. The total costs were computed by multiplying resource data by cost prices. The difference between total costs in the SEN group and the HO group were consequently computed and the 95% confidence interval (95% CI) for this difference was calculated. As cost data are typically skewed, confidence intervals for cost differences cannot be estimated with conventional methods that assume normality. To avoid distributional assumptions, we applied the non-parametric bootstrap [19, 20]. Basically, in the non-parametric bootstrap, samples of the same size as the original dataset are drawn by sampling with replacement from the observed data. These bootstrap samples can be used to estimate standard errors and confidence intervals. To obtain 95% confidence intervals for cost differences, we performed a non-parametric bootstrap with a 1000 replications .
For the cost-effectiveness analysis, the difference in total cost between the two treatment groups was compared with the difference in sensitivity and specificity. In other words, the increase (or decrease) in total costs per missed diagnosis (false positives and negatives) for the SEN group versus the HO group was calculated and constitutes the cost-effectiveness ratio.
SENs are capable of assessing and treating patients with an ankle or foot trauma at least as accurate as HOs. The percentage of false positives and false negatives in the SEN group was 7.4% compared with 8.6% in the HO group. For false negatives alone, these percentages were 2.9% and 4.7%, respectively. These differences were not statistically significant. Moreover, SENs accomplished these results with better patient satisfaction results than did the HOs. Furthermore, waiting times were decreased for patients with this type of injury. Full details on the clinical outcomes are presented in a separate paper .
Table 1 summarizes the time measurements. No significant differences were found between treatment groups with regard to the invested time of either SEN or HO. Also, as expected, the waiting time for patients between clinical assessment and radiograph (if made) were not different between groups. This waiting period depends on the pressure of activities at the radiology department and is in no way associated with the observer group. Finally, the time spent in the waiting room of the ED and the time spent between hearing the diagnosis and receiving final treatment (eg, application of pressure bandage) was significantly longer in the HO group.
Mean (SD) healthcare utilisation per treatment group
(n = 261)
(n = 249)
Health care Resource
Duration of Crutch-use (days)
Lower extremity cast
Number of hospital admittances
Duration of hospitalization (days)
Duration of operation (min)
Invested time patient
Time spent in waiting room (min)
Mean (SD) costs for treatment group for all patients
(n = 262)
(n = 249)
Difference (95% CI)†
HEALTH CARE SECTOR
2.7 (-0.3 ; 5.9)
0.3 (-0.5 ; 1.2)
-0.3 (-0.9; 0.4)
-0.6 (-3 ; 2)
Lower extremity cast
2.0 (-2.3; 6.3)
Costs related to operation
15.2 (-33; 65)
6.0 (-69 ; 81)
Valuation of time of SEN or HO
INDIRECT HEALTH CARE COSTS
Time spent in waiting room
-1.9 (-2.6 ; -1,2)
Waiting time assessment – radiograph
Waiting time diagnosis – treatment
Total health care costs
36 (-50 ; 123)
Total indirect health care costs
-3 (-4 ; -2)
33 (-84 ; 155)
Mean costs and effects by treatment group for all patients (missing data imputed)
SEN (n = 242)
HO (n = 233)
False positives and false negatives
Mean cost and effect differences between treatment groups and cost-effectiveness ratios for all patients (missing data imputed)
Cost difference †
Effect difference †
Incremental Cost-effectiveness ratio
Decrease in false positives and false negatives
Decrease in false negatives
Differences in total costs between the SEN group and HO group were not statistically significant. In daily practice, this means that we seem to have found an attainable solution to ED crowding that reduces waiting times, increases patient satisfaction and reduces workload for emergency physicians without increasing the costs. SEN cost only € 27 per false positive or negative that is avoided. The small difference in total costs in favour of the HO group is largely attributable to the slight (non-significant) difference in number of hospital admittances/operations, that accounts for larger costs in the SEN group. These costs are random since hospital admittance/operation is not a decision made by the observer, though is made by a supervising surgeon. Furthermore, the cost difference could partly be attributed to the costs of educating SENs. These costs are overestimated, since most of these costs were made for developing the educational programme and are not needed when a second group of nurses is trained. In addition, once a nurse has received training, the more patients he or she treats, the lower are the additional costs per patient.
Only few earlier publications on the costs and cost-effectiveness of alternative solutions to the problem of ED crowding are available. In the case of ANPs/PAs, one important publication revealed that ANP/PA delivered care was associated with higher costs than the standard care provided . In light of these findings by Sakr et al., it seems that the SEN concept might constitute an attractive alternative to the currently available midlevel practitioners. SENs provide the benefits of deploying nurses to do a physicians' job as do ANPs/PAs: shorter waiting periods, decreased workload for emergency physicians and increased patient satisfaction while maintaining excellent diagnostic accuracy . However, the costs associated with deploying SENs seem to be lower than those associated with the deployment of ANPs/PAs. The latter are more expensive because of the higher wages and costly educational programmes involved. Furthermore, it is important to note that the same results would have been seen when HOs/Emergency Physicians would be more expensive. This is due to the fact that the costs associated with income of the observer constitute only a small part of the total costs and therefore are irrelevant to the overall costs and consequently the cost difference.
Strengths and limitations of the study
For a critical appraisal of the design of the study, we refer to the clinical paper . With respect to the collection of information on utilization of some resources and (waiting/assessment) times, some limitations should be mentioned. Most resources were collected for all patients. However, as for crutch use, these data were collected for the last 115 patients and imputed for the rest of the groups. With regard to invested times by patients, the overall time spent in the ED as well as time spent in the waiting room were collected for all patients. However, waiting times between hearing the diagnosis and final treatment were collected for a limited number (N = 18) of patients per group and then imputed for both observer groups. As for the invested assessment times by the observers, these times were all recorded in a limited number of patients (N = 18) and then imputed for both observer groups. When imputing the mean values from the mentioned data, the variance was ignored. Obviously, it would have been better if we had collected these data for all patients. However, the sensitivity analysis performed later, resulted in similar results and therefore, the method used seems justified. Furthermore, since the effect is an underestimation of the variance, an increase in the variance will only affirm the observation that cost differences between the groups are not significant and therefore will not influence the conclusions drawn.
For the remaining resource variables, very few data were missing and considering the large sample size, estimates are considered to be accurate. With over 500 patients being included, the study is well-powered. Finally, all data were collected prospectively, contributing to the strength with which conclusions can be drawn from the presented results.
Considering the results of this study and keeping in mind the results of the accompanying clinical trial, costs and effects are equal for both the SEN group and the HO group. SENs cost only € 27 per false positive or negative and only € 18 per false negative that is avoided. Considering the benefits of the SEN-concept in terms of decreased workload for the physicians working in the ED, the increased patient satisfaction and shorter waiting times in the SEN group, SENs appear to be an attractive solution to the problem of ED crowding.
- Van Geloven AA, Luitse JS, Simons MP, Volker BS, Verbeek MJ, Obertop H: Emergency medicine in the Netherlands, the necessity for changing the system: results from two questionnaires. Eur J Emerg Med. 2003, 10: 318-322. 10.1097/00063110-200312000-00015.View ArticlePubMedGoogle Scholar
- Singh S: Self referral to accident and emergency department: patients' perceptions. BMJ. 1988, 297: 1179-1180.View ArticlePubMedPubMed CentralGoogle Scholar
- Dealey C: Emergency nurse practitioners: should the role be developed?. Br J Nurs. 2001, 10: 1458-1468.View ArticlePubMedGoogle Scholar
- Sakr M, Angus J, Perrin J, Nixon C, Nicholl J, Wardrope J: Care of minor injuries by emergency nurse practitioners or junior doctors: a randomised controlled trial. Lancet. 1999, 354: 1321-1326. 10.1016/S0140-6736(99)02447-2.View ArticlePubMedGoogle Scholar
- Ganapathy S, Zwemer FL: Coping with a crowded ED: an expanded unique role for midlevel providers. Am J Emerg Med. 2003, 21: 125-128. 10.1053/ajem.2003.50030.View ArticlePubMedGoogle Scholar
- Cooper MA, Lindsay GM, Kinn S, Swann IJ: Evaluating Emergency Nurse Practitioner services: a randomized controlled trial. J Adv Nurs. 2002, 40: 721-730. 10.1046/j.1365-2648.2002.02431.x.View ArticlePubMedGoogle Scholar
- Murray MK: The nursing shortage. Past, present, and future. J Nurs Adm. 2002, 32: 79-84. 10.1097/00005110-200202000-00005.View ArticlePubMedGoogle Scholar
- Mandrell BN, Hobbie W, Deatrick J, Lipman T, Hinds PS: When recruiting advance practice nurses is costly, develop your staff nurses. J Nurs Adm. 2004, 34: 543-545.View ArticlePubMedGoogle Scholar
- Stiell IG, Greenberg GH, McKnight RD, Nair RC, McDowell I, Worthington JR: A study to develop clinical decision rules for the use of radiography in acute ankle injuries. Ann Emerg Med. 1992, 21: 384-390. 10.1016/S0196-0644(05)82656-3.View ArticlePubMedGoogle Scholar
- Stiell IG, Greenberg GH, McKnight RD, Nair RC, McDowell I, Reardon M, Stewart JP, Maloney J: Decision rules for the use of radiography in acute ankle injuries. Refinement and prospective validation. JAMA. 1993, 269: 1127-1132. 10.1001/jama.269.9.1127.View ArticlePubMedGoogle Scholar
- Pigman EC, Klug RK, Sanford S, Jolly BT: Evaluation of the Ottawa clinical decision rules for the use of radiography in acute ankle and midfoot injuries in the emergency department: an independent site assessment. Ann Emerg Med. 1994, 24: 41-45.View ArticlePubMedGoogle Scholar
- Auleley GR, Kerboull L, Durieux P, Cosquer M, Courpied JP, Ravaud P: Validation of the Ottawa ankle rules in France: a study in the surgical emergency department of a teaching hospital. Ann Emerg Med. 1998, 32: 14-18. 10.1016/S0196-0644(98)70093-9.View ArticlePubMedGoogle Scholar
- Bachmann LM, Kolb E, Koller MT, Steurer J, ter Riet G: Accuracy of Ottawa ankle rules to exclude fractures of the ankle and mid-foot: systematic review. BMJ. 2003, 326: 417-10.1136/bmj.326.7386.417.View ArticlePubMedPubMed CentralGoogle Scholar
- Derksen RJ, Bakker FC, Geervliet PC, ES LK, Heilbron EA, Veenings B, Patka P, Haarman HJ: Diagnostic accuracy and reproducibility in the interpretation of Ottawa ankle and foot rules by specialized emergency nurses. Am J Emerg Med. 2005, 23: 725-729. 10.1016/j.ajem.2005.02.054.View ArticlePubMedGoogle Scholar
- Derksen RJ, Bakker FC, Heilbron EA, Geervliet PC, Spaans IM, ES LK, Veenings B, Patka P, Haarman HJ: Diagnostic accuracy of lower extremity X-ray interpretation by 'specialized' emergency nurses. Eur J Emerg Med. 2006, 13: 3-8. 10.1097/00063110-200602000-00002.View ArticlePubMedGoogle Scholar
- Derksen RJ, Bakker FC, ES LK, Spaans IM, Heilbron EA, Veenings B, Haarman HJ: Specialized emergency nurses treating ankle and foot injuries: a randomized controlled trial. Am J Emerg Med. 2007, 25: 144-151. 10.1016/j.ajem.2006.06.011.View ArticlePubMedGoogle Scholar
- Sakr M, Kendall R, Angus J, Sanders A, Nicholl J, Wardrope J, Saunders A: Emergency nurse practitioners: a three part study in clinical and cost effectiveness. Emerg Med J. 2003, 20: 158-163. 10.1136/emj.20.2.158.View ArticlePubMedPubMed CentralGoogle Scholar
- Oostenbrink JB, Koopmanschap MA, Rutten FF: Standardisation of costs: the Dutch Manual for Costing in economic evaluations. Pharmacoeconomics. 2002, 20: 443-454. 10.2165/00019053-200220070-00002.View ArticlePubMedGoogle Scholar
- Barber JA, Thompson SG: Analysis of cost data in randomized trials: an application of the non-parametric bootstrap. Stat Med. 2000, 19: 3219-3236. 10.1002/1097-0258(20001215)19:23<3219::AID-SIM623>3.0.CO;2-P.View ArticlePubMedGoogle Scholar
- Thompson SG, Barber JA: How should cost data in pragmatic randomised trials be analysed?. BMJ. 2000, 320: 1197-1200. 10.1136/bmj.320.7243.1197.View ArticlePubMedPubMed CentralGoogle Scholar
- Briggs AH, Wonderling DE, Mooney CZ: Pulling cost-effectiveness analysis up by its bootstraps: a non-parametric approach to confidence interval estimation. Health Econ. 1997, 6: 327-340. 10.1002/(SICI)1099-1050(199707)6:4<327::AID-HEC282>3.0.CO;2-W.View ArticlePubMedGoogle Scholar
- Briggs A, Fenn P: Confidence intervals or surfaces? Uncertainty on the cost-effectiveness plane. Health Econ. 1998, 7: 723-740. 10.1002/(SICI)1099-1050(199812)7:8<723::AID-HEC392>3.0.CO;2-O.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2474/8/99/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.