This article has Open Peer Review reports available.
Are online symptoms checkers useful for patients with inflammatory arthritis?
© The Author(s). 2016
Received: 25 March 2016
Accepted: 29 July 2016
Published: 24 August 2016
Online symptom checkers are increasingly used by patients however there is little published evidence of their effectiveness in real patients. The aim of this study was to evaluate how patients with inflammatory arthritis and inflammatory arthralgia use the internet to look for health information and to assess the advice given and diagnoses suggested by the NHS and WebMD symptom checkers in relation to the patients’ actual diagnoses.
Thirty-four patients with inflammatory arthritis (rheumatoid arthritis (n = 13), psoriatic arthritis (n = 4), unclassified arthritis (n = 4)) and inflammatory arthralgia (n = 13) newly presenting to a secondary care based clinic were identified using a consecutive sampling approach. Consenting patients were asked questions about their internet use in relation to their presenting symptoms. They then completed the NHS and the WebMD symptom checkers and their answers and the outcomes were recorded.
Sixteen patients had previously consulted the internet regarding their symptoms. Neither age nor gender significantly influenced internet usage. Actions advised via the NHS symptom checker were: call an ambulance (n = 11), attend A&E (n = 4), contact your GP straight away (n = 2), see your GP today (n = 6), or see your GP within 36 h (n = 11). The 5 most common differential diagnoses given by Web MD were gout (n = 28), rheumatoid arthritis (n = 24), psoriatic arthritis (n = 22), osteoarthritis (n = 18) and finger dislocation (n = 10). The most common first differential diagnosis was osteoarthritis (n = 12). Only 4 out of 21 patients with inflammatory arthritis were given a first diagnosis of rheumatoid arthritis or psoriatic arthritis.
Our data highlight that help seeking advice given online is often inappropriate and that the diagnoses suggested are frequently inaccurate. Recommendations to seek emergency advice may cause inappropriate healthcare utilization.
The early introduction of disease modifying anti-rheumatic drugs (DMARDs) leads to significantly improved outcomes for patients with rheumatoid arthritis (RA); in particular the first 12 weeks after symptom onset represent an important therapeutic window . Despite this, delays still occur between symptom onset and the initiation of therapy [2, 3]. In particular, patients often wait for prolonged periods before seeking advice from healthcare professionals [2, 3]. Reasons for patient delay include a lack of knowledge about inflammatory arthritis and available therapies, a normalisation of symptoms and attribution to external factors [4–6].
Health information is increasingly obtained via the internet . For example, Europe’s most popular health website, NHS choices , received over 10 million visits per week between January and October 2015 . One way in which RA patients might try to determine whether they need to seek medical attention for the early symptoms of RA is by seeking information online. Indeed a recent UK survey of patients with newly presenting RA and unclassified arthritis showed that 37 % consulted the internet about their symptoms before seeking help from their GP (manuscript submitted).
Symptom checker type websites include triage tools which direct the user toward the most appropriate source of health care, and self-diagnosis tools which provide the user with a specific diagnosis or series of differential diagnoses. The NHS symptom checker and the Boots WebMD symptom checkers are well known and widely used online resources. In December 2014 they were the two highest ranked tools in the Google, Yahoo and Bing search engines when the search terms ‘symptoms checker’ were entered. Furthermore in December 2014, the NHS symptom checker received 721,517 visits (personal communication from Health and Social Care Information Centre) and the Boots WebMD website received 422,300 visits .
These types of diagnostic online tools are marketed as a way of saving patients time, decreasing anxiety and allowing patients to take control of their own health . Many of these tools are designed to confer credibility by the use of symbols associated with the medical profession (e.g. stethoscopes) or by titles which infer that they can be used as proxies for doctors .
A recent study using 45 standardised patient vignettes to evaluate the accuracy of 23 symptom checkers found that they provided the correct diagnosis first on 34 % of occasions and listed the correct diagnosis within the top 20 differential diagnoses 58 % of the time. WebMD specifically provided the correct diagnosis 36 % of the time and listed it the top 20 62 % of the time . However, in that study, researchers inputted data into the symptom checkers based on information within the vignettes; whether patients would have interpreted their symptoms and answered the questions in the same manner as researchers did is unclear. WebMD was evaluated using patients in an ENT clinic where it provided the correct diagnosis first in only 16 % of cases but included the correct diagnosis in the list in 70 % of cases .
Research evaluating web-based tools typically use a range of methods. For example a study looking at the usability of ANSWER, a patient decision aid, used rating scales and thought verbalisations completed by the patients with RA and observational data provided by the researchers . Another study using ‘think-aloud’ protocols looked at how older adults search for information online and interpret symptoms using Google and WebMD’s symptom checker . In that study, participants received a vignette depicting symptoms of illness and were asked to talk out loud about their thoughts and behaviour while they were trying to diagnose the problem using internet tools.
The diagnostic accuracy of symptom checkers in relation to the symptoms of inflammatory arthritis and arthralgia has not been widely evaluated. The current study used patients with inflammatory arthritis or inflammatory arthralgia rather than symptom vignettes to test both the NHS Choices symptom checker and the Boots WebMD symptom checker and evaluated the advice given in relation to the patient’s actual clinical diagnosis.
Newly presenting patients with either clinically apparent synovitis or a new onset of symptoms consistent with inflammatory arthritis but without clinically apparent synovial swelling (from here on referred to as inflammatory arthralgia) were recruited from a secondary care based rheumatology clinic between the 9/2/15 and the 27/7/15. All eligible patients who attended clinic between these dates were approached to participate.
Assessment of internet use in relation to symptoms
Patients were asked if they had previously used the internet to find information about their presenting symptoms, which sites they had accessed and whether they had used symptom checkers.
Assessment of the NHS and the Boots WebMD checker
Both the NHS and the Boots WebMD checker begin by requesting demographic data. The NHS symptom checker then asks patients to select a category that best describes their symptoms and goes on to ask a series of multiple choice questions. The questions asked depend on the answers to previous questions therefore not all patients are asked the same questions. At the end, patients are advised where they should seek healthcare. WebMD features a pictorial representation of the human body, and patients are asked to identify the areas where their symptoms are located. They are then invited to identify relevant symptoms (e.g. joint pain or swelling) and, depending on the symptoms selected, a series of multiple choice questions are asked. At the end, a list of differential diagnoses is produced.
Patients were asked to complete the NHS and Boots WebMD symptom checkers and their answers and the outcomes given by the symptom checkers (help-seeking advice given by the NHS symptom checker and the 5 most likely diagnoses identified by Web MD) were recorded. The advice given or diagnoses suggested by the symptom checkers were compared with classified diagnoses made at the patient’s assessment in secondary care.
Data routinely collected during the new patient clinic included demographic variables, swollen and tender joint counts, DAS28 score, ESR, CRP, rheumatoid factor (RF) anti-CCP antibody status and rheumatological diagnosis according to established classification criteria. Autoantibody levels were considered to be ‘negative’ when they were below the upper limit of the normal reference range (ULN), ‘low’ when above the ULN but less than 3 times the ULN, and ‘high’ when over 3 times the ULN.
Median and interquartile ranges were calculated for the continuous variables, and groups were compared using non-parametric tests. Differences between proportions were assessed with chi-squared tests. Significance was determined when p < 0.05. Data were analysed using SPSS version 20.
Characteristics of participants. Summary values are shown as median (interquartile range)
p for difference between groups
p = 0.27
p = 0.13
Tender joint count (out of 28)
p = 0.12
Swollen joint count (out of 28)
p < 0.001
Patient global score (out of 100)
p = 0.38
p = 0.28
p = 0.22
p = 0.016
Rheumatoid factor positivity
p = 0.13
Anti-CCP antibody positivity *
p = 0.092
Previous use of the internet to gather information about symptoms
Sixteen patients (47 %) had previously consulted the internet regarding their symptoms. There was no significant effect of age (median age 44.5 vs 55.5 years for patients who had vs patients who had not consulted the internet p = 0.164, Mann–Whitney U test) or gender (36 % of men and 52 % of women used the internet, p = 0.388, Chi square test) on internet usage. Two patients recollected having previously used NHS choices, and one recollected having previously used WebMD.
This study found that the help seeking advice and diagnoses given by the symptom checkers tested was frequently inaccurate. For example, the NHS symptom checker inappropriately suggested that nearly half of patients should seek advice from emergency services. Only 56 % of patients were advised to seek help from primary care which would have been the appropriate triage level for their condition. Whilst we recognise the need for symptom checkers to try and rule out conditions requiring urgent medical treatment, were patients with inflammatory arthritis to follow the advice from the symptom checker it would result in poor use of emergency healthcare resources.
Previous studies have also suggested that symptom checkers tend to be risk adverse either advising patients to seek medical care when self-care would be appropriate or advising patients to attend hospital rather than consult a primary care physician . A study which used patient vignettes to test 15 different symptom checkers which gave triage advice found appropriate triage advice was only being given in 57 % of cases .
The diagnostic information provided by WebMD was mixed. Whilst only 4 out of 21 patients with inflammatory arthritis were given a first diagnosis of PsA or RA, these diagnoses were listed in the top 5 differentials in 15 out of 21 patients. However, osteoarthritis was suggested as the first differential diagnosis in 8 patients with inflammatory arthritis (5 with RA, 2 with PsA and 1 with an unclassified arthritis). Given that self-management approaches are frequently promoted as initial management strategies for patients with osteoarthritis this may delay appropriate help seeking for these patients with inflammatory arthritis.
Strengths and limitations
This study only evaluated symptom checker use in patients with inflammatory joint disease therefore our conclusions may not be generalizable to patients with other conditions. Although the sample size is small, the methodology of the study was more robust than many studies which have evaluated symptoms checkers as it used real patients rather than vignettes.
The use of online resource by patients in the context of their symptoms is commonplace and likely to become more frequent over time. This study highlights that, in the context of inflammatory arthritis symptom, triage advice from the NHS Choices and WedMD checkers is risk averse and often inappropriate. Whilst the accuracy of the first suggested diagnosis is poor the list of differentials does often contain the correct diagnosis.
ACL, anterior cruciate ligament; CCP, cyclic citrullinated peptide; CREST, Calcinosis, Raynaud’s phenomenon, Oesophageal dysmotility, Sclerodactyly, and Telangiectasia; CRP, C reactive protein; DAS 28, disease activity score 28 joints using erythrosedimentation rate; DMARD, disease modifying antirheumatic drug; ESR, erythrocyte sedimentation rate; OA, osteoarthritis; PsA, psoriatic arthritis; RA, rheumatoid arthritis; RF, rheumatoid factor; SLE, systemic lupus erythematosus; ULN, upper limit of normal
This study received no funding.
Availability of data and materials
Data files with participants' clinical data and symptom checker answers are available as part of the Additional file 2.
LEP and GM contributed equally to this study. LEP was involved in the design of the study, data collection, interpretation of the data and drafting the manuscript. GM was involved in data collection, statistical analysis and critical revision of the manuscript. GS was involved in critical revision of the manuscript. KR was involved in conception and design of the study, interpretation of the data and critical revision of the manuscript. All authors gave final approval of the version to be published.
The authors declare that they have no competing interests.
Ethics approval and consent to participate
All patients approached to take part in the current survey had agreed, on the same visit to take part in a research study addressing predictors of outcome in early arthritis. That particular study had been approved by The Back Country Research Ethics Committee and all patients gave written informed consent to participate. The current survey of the performance of internet based resources was approved by Sandwell and West Birmingham Hospitals NHS Trust’s Research and Development department as a service evaluation and participating patients consented to that.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Van der Linden MPM, Le Cessie S, Raza K, et al. Long term impact of delay in assessment of patients with early rheumatoid arthritis. Arthritis Rheum. 2010;62:3357–546.Google Scholar
- Kumar K, Daley E, Carruthers DM, et al. Delay in presentation to primary care physicians is the main reason why patients with rheumatoid arthritis are seen late by rheumatologists. Rheumatology. 2007;46:1438–40.View ArticlePubMedGoogle Scholar
- Raza K, Stack R, Kumar K, et al. Delays in assessment of patients with rheumatoid arthritis: variations across Europe. Ann Rheum Dis. 2011;70:1822–5.View ArticlePubMedGoogle Scholar
- Sheppard J, Kumar K, Buckley CD, Shaw KL, Raza K. ‘I just thought it was normal aches and pains’: a qualitative study of decision-making processes in patients with early rheumatoid arthritis. Rheumatology (Oxford). 2008;47:1577–82.View ArticleGoogle Scholar
- Kumar K, Daley E, Khattak F, Buckley CD, Raza K. The influence of ethnicity on the extent of, and reasons underlying, delay in general practitioner consultation in patients with RA. Rheumatology (Oxford). 2010;49:1005–12.View ArticleGoogle Scholar
- Stack RJ, Shaw K, Mallen C, Herron-Marx S, Horne R, Raza K. Delays in help seeking at the onset of the symptoms of rheumatoid arthritis: a systematic synthesis of qualitative literature. Ann Rheum Dis. 2012;71:493–7.View ArticlePubMedGoogle Scholar
- Christmann, S. Health literacy and Internet. Recommendations to promote Health Literacy by the means of the Internet. 2005. Available from: http://eurohealthnet.eu/sites/eurohealthnet.eu/files/publications/pu_8.pdf. [Accessed 21 Dec 2015].
- NHS Choices. NHS Choices Annual Report 2012/13. NHS Choices; 2013. Available from:http://www.nhs.uk/aboutNHSChoices/professionals/developments/Documents/annual-report/nhs-choices-annual-report-2012.pdf. [Accessed 21 Dec 2015].
- Masefield N. NHS Choices weekly website report - week 42. NHS Choices; 2015. Available from: http://blogs.nhs.uk/choicesblog/2015/10/27/nhs-choices-weekly-website-report-week-42-2/. [Accessed 21 Dec 2015].
- Compete PRO Millward Brown Digital Available from: https://siteanalytics.compete.com/webmd.boots.com/#.VniF4hWLTIU [Accessed 21 Dec 2015].
- Lupton D, Jutel A. ‘It’s like having a physician in your pocket!’ A critical analysis of self-diagnosis smartphone apps. Soc Sci Med. 2015;133:128–35.View ArticlePubMedGoogle Scholar
- Semigran HL, Linder JA, Gidengil C, Mehrotra A. Evaluation of symptom checkers for self diagnosis and triage: audit study. BMJ. 2015;351:h3480.View ArticlePubMedPubMed CentralGoogle Scholar
- Farmer SE, Bernardotto M, Singh V. How good is Internet self-diagnosis of ENT symptoms using Boots WebMD symptom checker? Clin Otolaryngol. 2011;36:517–8.View ArticlePubMedGoogle Scholar
- Li LC, Adam PM, Townsend AF, et al. Usability testing of ANSWER: a web-based methotrexate decision aid for patients with rheumatoid arthritis. BMC Med Inform Decis Mak. 2013;13:131.View ArticlePubMedPubMed CentralGoogle Scholar
- Luger TM, Houston TK, Suls J. Older adult experience of online diagnosis: results from a scenario-based think-aloud protocol. J Med Internet Res. 2014;16(1):e16.View ArticlePubMedPubMed CentralGoogle Scholar