A study was conducted for a month (March 2018) in the Department of Physical Therapy at Clifton Hospital and Institute of Physical Medicine and Rehabilitation, Dow University Hospital, Karachi Pakistan.
Study aim
The aim of the study was to develop and validate a novel self-reporting scale named as the ‘General Rehabilitation Adherence Scale’, in Urdu language to measure adherence to physical therapy, rehabilitation and exercise in Pakistani patients with musculoskeletal disorders.
Recruitment and randomization
The study used the block randomization technique. The data collection was conducted daily from 9 am to 1 pm in the day and from 3 pm to 9:30 pm in the evening. We used the patients’ appointment numbers to randomize the sample. Patients with an even-numbered appointment were invited to participate in the study and odd-numbered appointments were left out. The sequence was reversed the other day, i.e., odd-numbered appointments were invited. A computer-generated list was used for this purpose. This approached helped in eradication of bias in selection.
Participants and eligibility criteria
Patients who had to undergo physical therapy for any musculoskeletal condition for at least 2 weeks were invited to participate in the study. The study included both male and female patients who were adult and above 18 years. Patients were asked to provide their written consent before handing the questionnaire. Those patients who were not willing to participate were left out. Incomplete questionnaires were not included in the study.
Sampling technique and sample size
We conducted random sampling and the sample size was calculation was based on item response theory, i.e., item-to-respondent ratio. The questionnaire contained eight items and therefore an item-to-respondent ratio of 1:20 was considered enough for validation purpose. The required sample size was calculated by the following formula:
Where, N = sample size; ni = number of items in questionnaire and R = item-to-respondent ratio. The sample size obtained was 160. A drop-out rate of 10% which was roughly 20, was added to yield final sample size of 180 patients.
Research instrument development and conceptualization
A novel research tool named as the General Rehabilitation Adherence Scale (GRAS) was formulated and used for this study. Prior to instrument development, a thorough literature review was carried out [3, 7]. The tool was developed in Urdu language since it was the local language of Pakistanis. The initial draft of the tool contained 13 items related to adherence regarding physiotherapy. All items were multiple choice questions (MCQs) and were graded. The draft was subjected to review by a panel of experts and was validated with eight items.
Face and content validity
The initial draft was subjected to face and content validity. A panel of experts was formed that comprised of two pharmacists, two physiotherapists, a rheumatologist, an occupational therapist, two academicians and a social scientist. The panel carried out face and content validity of GRAS. In addition, patients were also consulted in the process. Face validation was conducted through Delphi consensus method [25]. The content validity index (CVI) and ratio (CVR) were calculated by asking every expert to mark each item in the questionnaire as essential or non-essential, from the perspective of a patient [26, 27]. The content validity ratio was calculated using the following formula:
$$ CVRi=\frac{n\raisebox{1ex}{$e-N$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}{\raisebox{1ex}{$N$}\!\left/ \!\raisebox{-1ex}{$2$}\right.} $$
Where, CVRi is the content validity ratio of each item; ne was the number of experts indicating essentiality of item and N is the total number of experts in the panel. Content validity index (CVI) was analysed by calculating mean ratio of the tool using the following formula.
$$ CVI=\left[ CVR1+ CVR2+ CVR3+\dots CVR13\right]/13 $$
Determination of cut-off values
Receiver operating curve (ROC) analysis was used to determine cut-off value for the scale [28]. The focus of ROC analysis was to determine a score that had highest sensitivity and lowest inverse of specificity [29]. The scoring of GRAS was later defined based on cut-off values.
Patient adherence levels and scoring criteria
Based on cut-off value, we defined adherence to rehabilitation as patient’s concordance to the physical therapy schedule and categorized it into five levels, i.e., high, good, partial, low and poor. The scoring criterion was defined along with the levels of adherence. The final content validated draft of the GRAS had a total of eight items that awarded a maximum score of 24. Each item of the tool had four possible options namely, always, mostly, sometimes and never, that awarded an individual score of 0, 1, 2 and 3 respectively. A patient would be categorized as being ‘highly’ adherent if his score is between 20 and 24. Similarly, patients would be classified as being ‘good’ in adherence if the final score is between 17 and 19 and ‘partial’ if the score is between 12 and 16. A patient would be considered having a ‘low’ adherence if the score is between 8 and 11 and ‘poor’ if the score is 7 or less. The Urdu and English versions of GRAS and its scoring code are available as Additional files 1 and 2.
Factor analyses
The factor structure of GRAS was examined through Exploratory factor analysis (EFA) using Principle component analysis (PCA) with Varimax rotation. It was carried out with an item-to-respondent ratio of 1:20. The model was then confirmed through Partial confirmatory factor analysis (PCFA) using Maximum likelihood analysis with same rotation. The incremental fit indices namely the comparative fit index (CFI), normed fit index (NFI) and Tucker Lewis index (TLI) were calculated. Additionally, absolute fit index, i.e., root mean square error of approximation (RMSEA) was reported [30, 31].
Internal consistency and reliability analyses
The internal consistency was calculated by test-retest method using Cronbach’s alpha (α) values. The value was considered satisfactory if it was ≥0.5. Besides, intra-class correlation (ICC) was also calculated [5, 30]. Alternatively, the McDonald’s coefficient (ωt) was also calculated as an estimate of reliability [32, 33]. The test-retest reliability between two time-points was assessed after a gap of 4 weeks through Pearson’s correlation coefficient (ρ). A value of (ρ) more than 0.75 and p-value < 0.05 was considered significantly strong correlation [30, 31].
Known group validation
We believed that educated patients would be more adherent to their prescribed PT schedule as compared to their uneducated counterparts. The known group validity was evaluated through chi square (χ2) test and a p-value less than 0.05 was considered acceptable.
Sensitivity analysis
Sensitivity analysis was conducted to determine the GRAS’s sensitivity to identify adherent patients correctly, as well as its specificity, i.e., to accurately distinguish non-adherent patients. Sensitivity, specificity, accuracy and predictive values were reported as a percentage (%) and in 95% confidence interval range. Likelihood ratios were reported as a value and in 95% confidence interval range. Standard logit confidence interval method was used to report predictive values [28, 34]. Log method was used for determination of likelihood ratios and Clopper-Pearson method was used to calculate sensitivity, specificity and accuracy of the scale [35].
Questionnaire administration
The GRAS questionnaire was handed to patients after obtaining their consent. Patients filled in their responses returned the questionnaire. Apart from GRAS, the patients were also provided with a demographic questionnaire that contained questions related to age, gender, education, occupation, income, residence, health insurance and comorbidity. A second copy of the scale was filled by patients at 2nd timepoint. The three forms, i.e., demographic questionnaire and, GRAS at 1st and 2nd timepoints were kept together in a separate file designated for a patient using his/her medical record number as identifier.
Data analysis
All data were analyzed through IBM SPSS version 22, Armonk, NY, USA. The demographic data was reported in sample counts (N) and percentages (%). The associations between demographic and adherence variables were tested by chi square (χ2) test for association. Statistical significance was considered at p-value less than 0.05.
Ethics approval and patient consent
The patients were explained about the objectives of the study and their written consent was sought. The study was approved by the Institutional Review Board of Allied Med Ethics (Ref# NOV:15) and Department of Physiotherapy, Clifton Hospital Karachi.