Skip to main content

Development and validation of a questionnaire to determine medical orders non-adherence: a sequential exploratory mixed-method study



Patients’ non-adherence with medical orders of physicians in outpatient clinics can lead to reduced clinical effectiveness, inadequate treatment, and increased medical care expenses. This study was conducted to develop and validate a questionnaire to determine the reasons for patients’ non-adherence with physicians’ medical orders.


A sequential exploratory mixed-method study was conducted in two stages. The first stage comprised a qualitative stage to generate the primary items of the questionnaire. This stage provided findings of two sub-stages comprising a literature review and the findings of a qualitative conventional content analysis of 19 semi-structured interviews held with patients, physicians, and managers of the outpatient clinics in Kerman, an area located in southeastern Iran. The second stage comprised a quantitative study aiming evaluation of the instrument psychometric properties, including the face, content, construct, and reliability assessment of the questionnaire. Construct validity assessment was evaluated using exploratory factor analysis (EFA). The reliability assessment was done using assessing internal consistency (Cronbach’s alpha). To assess the construct validity of the questionnaire, four hundred and forty patients referred to outpatient clinics in Kerman were selected using stratified convenience sampling to fill out the questionnaire. The sample size was calculated using the Cochran formula. Qualitative and quantitative data were analyzed by MAXQDA 10 and Stata version 14, respectively.


The primary items contained 57 items, of which 42 met the minimum acceptable value of 0.78 for item-level content validity index (I-CVI = 1 for 24 items and I-CVI = 0.8 for 18 items). Item-level content validity ratio (I-CVR) was confirmed for 18 items with a minimum acceptable value of 0.99 for five experts. Finally, 18 items obtained the acceptable value for both I-CVI and I-CVR indicators and were confirmed. Using EFA, four factors (intrapersonal-psychological, intrapersonal-cognitive, provider-related, and socio-economic reasons) with 18 items and Cronbach’s alpha coefficient of 0.70, 0.66, 0.73, and 0.71, respectively, were identified and explained 51% of the variance. The reliability of the questionnaire (r = 0.70) was confirmed.


The questionnaire with four dimensions is a valid and reliable instrument that can help determine the perceived reasons for non-adherence with medical orders in the outpatient services system.

Peer Review reports


Problems related to patients’ non-adherence with medical orders are among the main concerns of the healthcare system. Suppose patients do not comply with therapeutic recommendations of physicians, despite the efforts of physicians. In that case, the desired therapeutic outcomes are not achieved, and it is the primary concern in the clinical field [1]. Hence there is a need to find an instrument to detect medical orders non-adherence.

Healthcare adherence is related to the individual’s ability to keep medical orders prescribed by physicians. It includes the timely presence in the predetermined treatment programs, the timely and proper use of medicines, follow-up of the disease for any later required referral, and adherence with health behaviors change. In healthcare non-adherence, patients do not comply with therapeutic recommendations consciously and intentionally [2].

World Health Organization (WHO) defines therapeutic adherence as “the extent to which a person’s behavior (in terms of taking medications, following diets, or executing lifestyle changes) coincides with medical or health advice” [3]. Otherwise, non-adherence occurs.

Non-adherence may lead to frustration for medical providers [4] and result in undesirable clinical outcomes, increased use of health services [5], increased rate of diseases and mortality [6], and waste of resources in healthcare systems [7].

Studies on various diseases have shown adverse outcomes of the non-adherence, such as the risk of death in HIV-infected patients [8], risk of mortality in patients suffering cancer [6], increase in the risk of diabetes [9], and medical costs for cancer, diabetes, and cardiovascular disease increase over twice [10]. Additionally, 10% of hospitalization in the elderly is due to medication non-adherence [10, 11]. Outpatient clinics are the first contact of patients and the community, and many hospital admissions are taking place through these clinics [12]. Non-adherence with therapeutic recommendations is one of the reasons for the disease’s progression and complications [13, 14], prolonged treatment period, and increased health care costs [15,16,17]. Effective interventions need to be developed to improve patients’ adherence and improve clinical outcomes.

There are various instruments to assess adherence and non-adherence in different diseases [18,19,20,21]. For example, Sidorkiewicz et al. developed an instrument to assess adherence for each drug taken by patients, usable in hospital and primary care settings [22]. Another study conducted in a developing country developed an approved instrument that measures medication adherence in patients with chronic diseases [23]. Another instrument for non-adherence was developed and validated, which was focused on intentional non-adherence [24]. However, existing instruments are disease and order specific, which does not include the most relevant of patients’ perceived reasons for non-adherence with medical orders generally in outpatient clinics, irrespective of disease, and prescription types. In this regard, to develop a valid and reliable instrument covering most relevant the perceived reasons for medical orders non-adherence can help the policy makers understand and identify the reasons and consequently adopt tailored policies to improve adherence [25, 26]. Thus, the present study aimed to develop a questionnaire and evaluate its psychometric properties to determine the perceived reasons for patients’ non-adherence with physicians’ medical orders in the outpatient clinics.


The present study was a mixed-method study with a sequential exploratory design. This study was conducted in two stages. The first stage aimed to conduct a qualitative study to generate a pool of the questionnaire’s primary items. This stage focused on findings of two sub-stages comprising a literature review and a qualitative conventional content analysis findings. The second stage followed a quantitative study to evaluate the psychometric properties of the instrument.

We held nineteen semi-structured interviews with patients, physicians, and managers of the outpatient clinics in Kerman, an area located in southeastern Iran. The second stage comprised a quantitative study aiming evaluation of the instrument psychometric properties, including the face, content, construct validity, and reliability assessment of the questionnaire. Construct validity assessment was evaluated using exploratory factor analysis (EFA).

Literature review and qualitative inquiry

Literature review and qualitative study were conducted to identify the reasons for non-adherence with medical orders and generally facilitate item generation [27]. We first conduct a literature review to identify the existing questionnaires and extract the reasons related to non-adherence with therapeutic orders prescribed by physicians from previous studies. Several valid scientific databases in Persian and English languages were aimed to retrieve the most relevant information about non-adherence with medical orders. The Persian databases of Scientific Information Database (SID), Iranian Magazines (Magiran), and Iranian Research Institute for Information Science and Technology (IranDoc) and English databases of Web Of Science (WoS), PubMed, Scopus, and google scholar were searched. The search strategy will include only terms relating to or describing non-adherence with medical orders. The most relevant keywords used were non-compliance, compliance, non-adherence, adherence, outpatient, ambulatory care, ambulatory care facilities, therapeutic recommendation, therapeutic orders, and medical orders. Definition of non-adherence and its dimensions and reasons were extracted, and many items were generated.

Furthermore, a qualitative study with a conventional content analysis approach was conducted to complete the reasons for non-adherence with medical orders in the setting of outpatient clinics. In this sub-stage, we conducted 19 semi-structured interviews with patients (n = 10), physicians (n = 5), and managers of the outpatient clinics (n = 4) in Kerman, an area located in southeastern Iran. Physicians, health care managers, and medical officials were selected using the purposeful sampling method, and patients were selected using convenience sampling. Qualitative data were analyzed using conventional content analysis by MAXQDA version 10. Qualitative data contribute to the enrichment and development of the concept and is a valuable resource for developing instruments [28]. More details of the results of the qualitative study were reported elsewhere [29]. The findings obtained from the literature review and the codes derived from the qualitative data analysis were mixed together. Eventually, a list of the most relevant reasons related to non-adherence with medical orders was identified. A first draft of the questionnaire was prepared, and included items were reviewed several times to ensure appropriate wording of the items, remove duplicates, and arrange their arrangement.

The questionnaire for non-adherence with therapeutic recommendations was scored based on a 5-point Likert scale (0 = very unimportant, 1 = unimportant, 2 = slightly important, 3 = important, 4 = very important). Patients at least 18 years old who experienced at least once non-adherence with medical orders during the past 4 weeks were considered the target group to fill out the questionnaire. The questionnaire was self-administered by patients. It consisted of 18 items, and the higher scores represented the higher importance of a given reason or factor. The questionnaires were completed anonymously and in a self-reported manner to overcome social desirability bias [30]. To minimize the end aversion bias, we tried to make questions clear for respondents.

Evaluation of the psychometric properties of the instrument

At this stage, the instrument’s psychometric properties, including the face, content, construct validity, and reliability (internal consistency), were assessed.

Face and content validity assessment

At this stage, the approval of experts for the content of the instrument was necessary. Therefore, after obtaining the experts’ approval for the instrument’s content and recording their comments, the instrument was drafted, the required changes in the instrument were made.

Content validity ratio (CVR) and content validity index (CVI) for each item and scale were calculated to determine the questionnaire’s content validity [26]. To determine the necessity and to include the most necessary items, the CVR, and to ensure the relevance, the CVI, was used [31]. The opinions of five experts with research backgrounds in questionnaire development and adherence and compliance in the health were surveyed. In the questionnaire given to the experts, it was explicitly explained which concept is measured by the questionnaire to give their opinion with enough knowledge about the subject.

The experts were asked to respond to 57 items based on scoring each item from 1 to 3 with a three-degree range of 1 = not necessary, 2 = useful but not essential, 3 = essential. Then, based on the experts’ responses, the CVR was determined using the following equation, and each item was scored and conformed with the Lawshe table for the number of experts involved.

$$ CVR=\left({N}_e-N/2\right)/\left(N/2\right) $$

Where Ne is the number of experts who rated the item as necessary, N is the total number of experts who rated the item. The acceptable value for the questionnaire’s validity is determined according to the Lawshe Table (1975) [32]. According to this table, in the case of involving five experts, items with a validity score higher than 0.99 were considered acceptable in this study.

The experts were asked to score 57 items based on a 4-point Likert scale (1 = completely irrelevant, 2 = partially relevant, 3 = greatly relevant, 4 = completely relevant) to determine the degree of relevance. The CVI was calculated by dividing the total number of experts that scored 3 or 4 by the total number of experts. The items with a CVI of 0.78 and higher were considered acceptable [33].

The face validity is an objective judgment about the construct of an instrument, which indicates the instrument’s relevance to the study’s aim, how to express phrases, the wording of questions, and understanding the researcher’s intended concept [34]. The experts were asked to give their opinions on the items’ content regarding the main question of the study and items’ wording (literary editing and fluency of terms and words) and, if necessary, express their suggested item. Also, for assessing the questionnaire’s comprehensibility by the target group, an interview was conducted with ten low-educated patients by one of the researchers to find the difficulty, the possibility of ambiguity, and inappropriate comprehension of the expressions.

Construct and factorial validity

Sampling, setting and data gathering

The sample size was calculated using Cochran’s formula for descriptive cross-sectional studies to measure the rate and perceived reasons for the non-adherence with physicians’ medical orders. A previous study was not found in Iran to measure generally the types of non-adherence with medical orders. Thus, the value of p and q was considered 0.5 (the state in which it gives the maximum variance), and z and d (margin of error) were regarded to be 1.96 and 0.05, respectively. The primary sample was calculated to be three hundred and eighty-five samples. As some questionnaires might be incompletely filled, 15 % (15%) was added to the primary sample size. So, the final sample size was considered to be four hundred and forty samples.

The patients were selected using a two-stage stratified convenience sampling method. First, the outpatient clinics by ownership type, including clinics affiliated with social security, armed force, medical university, private and non for profit, were considered strata. The number of clinics in each stratum was selected proportionally to the total number of clinics. Furthermore, considering socio-economic variation, the clinics were randomly selected across the geographical areas to cover the most advantaged areas to the most disadvantaged ones. Second, one of the researchers referred to sampled clinics, and four hundred and forty patients were selected based on convenient sampling during 1 month (4 weeks), twenty-five patients from each clinic. The questionnaires were self-administered and personally distributed by the researcher after explaining the research’s purpose to respondents. The face-to-face interview was conducted if the patients were illiterate.

Construct validity

Construct validity is the degree to which the instrument is consistent with the theory. It is measured and could be established in different ways, such as examining the new instrument’s correlation with a well-validated questionnaire, distinguishing one group from the other based on some crucial variables [35], different forms of factor analysis, and other statistical evaluations. We had no well-established scale measuring non-adherence in the Persian language, irrespective of disease and medical order types. Therefore, we intended to examine whether the scale “behaves as expected concerning known groups” (i.e., known-group comparison approach) [36]. The method of known groups is a typical method for supporting construct validity and is given when a test can discriminate between a group of people known to have a specific characteristic and a group who do not have the characteristic. The instrument’s total score was compared based on gender and insurance coverage [37,38,39,40].

To investigate the factorial structure of the questionnaire, the EFA was used. The principal component analysis (PCA) and varimax rotation were performed to conduct EFA. Assuming the factors are not correlated with each other and are independent, rotation of the orthogonal method and the standard method of varimax was used.

To confirm the hypothesis, EFA, Kaiser-Meyer-Olkin (KMO), and Bartlett’s test were performed. The KMO test is used in EFA to determine how suited the data is and measure the sampling adequacy. Bartlett’s test of sphericity is used to assess an adequate amount of correlation (p < 0.05) between items. Scree plot was used to determine the number of factors. The parallel analysis and scree plot were also conducted to determine the accurate number of factors compared with eigenvalues, which tend to overestimate the number of factors [41]. All data processes were conducted and analyzed using SPSS software version 16.0.

Reliability assessment

The reliability assessment of the dimensions was measured using internal consistency with Cronbach’s alpha. According to Nunnally and Bernstein (1994), newly developed instruments with Cronbach’s alpha of more than 0.5 are acceptable, but the threshold for other cases is 0.7 [42].

Ethical considerations

The Ethics Committee of Kerman University of Medical Sciences approved this study (Approval ID: IR.KMU.REC.1397.040). After explaining the study’s objectives to the participants, written informed consent forms were obtained from them, and they were ensured about the confidentiality of their information. All participants were told to withdraw from the study at any time.


In this section, the results are reported according to the steps of the study.

Stage one: literature review and qualitative inquiry

From the literature review and qualitative interviews analysis in the first stage, 30 and 48 items, respectively, were extracted. After removing duplicates and similar items, 57 items were obtained as items of the questionnaire’s initial version.

Stage two: evaluation of the psychometric properties of the instrument

At this stage, the analysis of the instrument’s psychometric properties, including content validity, formal validity, construct validity, and reliability (internal consistency), were reported.

Validity assessment

At this stage, after obtaining experts’ comments, CVR and CVI values were calculated to evaluate the content validity of the questionnaire, and 18 items achieved acceptable CVR and CVI values and were approved.

The instrument’s content validity results were as follows: of 57 items, the item content validity index (I-CVI) for 42 items was higher than the acceptable value of 0.78 (I-CVI = 1 for 24 items and I-CVI = 0.8 for 18 items). Item content validity ratio (I-CVR = 1) for 18 items was higher than the acceptable value of 0.99. The scale content validity ratio (S-CVR), and the scale content validity index (S-CVI) were also obtained. The S-CVR and S-CVI for the final 18 items were equal to 1.0 and higher than 0.78. Finally, 18 items obtained the acceptable value for both indicators and were confirmed.

In the stage of evaluation of formal validity, of 57 items, some items were underscored based on the experts’ opinion due to repetition, overlapping with other items, information gaps, and information asymmetry between patients or recipients of the health care service and health care system (health care provider) in response to the item. According to the suggestions by the experts about items’ wording and arrangement, changes were made.

We interviewed ten patients to evaluate the comprehensibility of the items of the questionnaire. According to their comments, some examples were added to some items to meet the fluency and better understanding.

Demographic characteristics

The questionnaire was distributed to four hundred and forty samples. Four hundred out of 440 questionnaires (response rate = 90.9%) was received entirely and returned. Among patients referred to outpatient clinics, 184 (46%) were male, and 216 (54%) were female. The age of most participants was 30 to 60 years. Most of the patients (70.75%) were married. About 13% of them had an academic education. About 50% of the patients were employed, and the remaining were unemployed or homemakers. Most patients (92.25%) were residents of Kerman province. Only about one-fourth of the patients were living in the center of the province. More than 90% of the participants had basic insurance, and only 11% had supplementary health insurance. In terms of economic status, almost half of the patients (53.55%) had an income between 10 and 20 million Rials (IRR). All participants had Iranian nationality (See Table 1).

Table 1 Descriptive statistics of demographic variables of patients referred to outpatient clinics in Kerman (n = 400)

Exploratory factor analysis

Kaiser-Meyer-Olkin (KMO) test for each item was higher than 0.5 with an overall value of 0.73 for the whole questionnaire, which is above 0.6, indicating that the sample size and factorability are met [43] for conducting EFA. The significance level of the Bartlett test Bartlett’s test of sphericity was less than 0.05. Therefore, it indicates an adequate amount of collinearity (p < 0.05) between items.

The scree plot with a four-factor structure with 18 items obtained after performing the PCA, considered the most suitable questionnaire structure (See Fig. 1). As shown in Table 2, the initial eigenvalues for all four factors are higher than one.

Fig. 1

Scree plot test based on EFA for patients’ non-adherence with medical orders in outpatient clinic settings. Note. EFA = exploratory factor analysis

Table 2 Statistical indices of exploratory factor analysis using principal components analysis

Approximately 51% of the total variance was explained by the four factors (factor 1 = 15.05%, factor 2 = 14.61%, factor 3 = 12.43%, and factor 4 = 9.04%). After rotation, items 3, 4, 7, and 8 were loaded with the first factor, items 10–15 were loaded with the second factor, items 1, 2, 5, 6, and 9 were loaded with the third factor, and items 16–18 were loaded with the fourth factor (Table 3). According to the items’ content in each factor, intrapersonal-psychological, provider-related, intrapersonal-cognitive, and socio-economic reasons seem to be appropriate as perceived reasons by patients for the factors.

Table 3 Rotated component matrix of each item using exploratory factor analysis with principal component analysis method

Construct validity

Our findings for convergence among the adherence score and its relationship with respondents/patients’ demographic variables showed a significant relationship between non-adherence with gender and insurance coverage (p < 0.05). Non-adherence score was higher in men in comparison with the Females. Furthermore, patients who lack basic health insurance coverage have a higher score of non-adherence than insured patients (p < 0.05). These results are consistent with the literature and results of other existing standard instruments that indicate a convergence between our instrument and other existing instruments.

Reliability assessment

Reliability assessment of the questionnaire (Average Cronbach’s alpha = 0.70) was confirmed. The Cronbach’s alpha coefficient for intrapersonal-psychological, intrapersonal-cognitive, provider-related, and socio-economic reasons was 0.70, 0.66, 0.73, and 0.71, respectively (Table 4). Cronbach’s alpha coefficient for intrapersonal-psychological reasons was less than the acceptable level of 0.7. This scale measures different characteristics to explain why the Cronbach’s alpha coefficient for this dimension was less than the acceptable level.

Table 4 Results of the reliability assessment using Cronbach’s alpha

The medium value of Cronbach’s alpha coefficient can be due to this finding that about 50% of the participants had identical responses. Our EFA showed four dimensions of the scale, which means four different characteristics were on this scale. This point could explain why the coefficient alpha was not higher than 0.7 as an acceptable level. Moreover, if the number of test items of each scale is too small, the scale’s reliability would be underestimated [44]. The low variability in the scale scores especially the intrapersonal-cognitive scale is another reason for the Cronbach alpha lower than 0.7, which means that about half of the subjects in this sample had similar values. If variability between scale scores were greater, which would arise in a population with varying levels of adherence, internal consistency would be strengthened [45]. In 3 items related to intrapersonal-psychological reasons, more than half of the participants selected the number of 0 or 1, which decreased Cronbach’s alpha coefficient. The internal consistency reliability can be improved by increasing the number of responses [45]. However, the average Cronbach’s alpha coefficient of the questionnaire 0.70 and is considered acceptable. The final version of the developed questionnaire was presented in the Supplementary file 1.


This study was conducted to design and evaluate the psychometric properties and factor structure of a questionnaire to determine the perceived reasons for patients’ non-adherence with therapeutic recommendations in outpatient clinics. The EFA findings explain about 51% of the variance, indicating four dimensions of intrapersonal-psychological, intrapersonal-cognitive, provider-related, and socio-economic reasons.

Intrapersonal-psychological reasons include hastiness and hasty judgment, others’ advice, anxiety, fear and anxiety for the consequences of the diagnosis and treatment, and experiences of the individuals. The intrapersonal-cognitive reasons include individual perceptions, individual attitudes toward disease, health literacy, patient preferences, and values. The provider-related reasons can include factors such as inappropriate physician behavior, adverse effects of prescriptions, and unnecessary diagnostic and the therapeutic measures. The socio-economic reasons include the costs, problems, and concerns of life-related to social issues, family support, and access to facilities and medicines.

A study conducted in Ghana concluded that knowledge and experience act as influential factors for non-adherence [46]. Moreover, other studies showed that the relationship between physician and patient [47], patient involvement in the treatment process [48, 49], social attention, and support such as family support in helping to maintain continuous care and treatment [50, 51] affect decision making and determination of reasons for non-adherence. A systematic review and meta-analysis in Ethiopia also identified fear and waiting time as determinants of non-adherence [52]. Studies in Tanzania [53] and Australia [54] have also reported economic factors as reasons for non-adherence.

Instruments developed in previous studies show that these studies evaluated the reasons for non-adherence to a specific disease. In a study by Chizzola et al. (1996), adherence with therapeutic recommendations in patients with cardiovascular disease in outpatient clinics was investigated using an instrument measuring only socio-economic factors and patients’ knowledge of medications [55]. In a 14-item questionnaire designed by Jank et al. (2009), individuals’ attitudes towards medicines, medical care expenses, and daily life obstacles were evaluated [56]. Many items evaluated in the study are consistent with two dimensions of intrapersonal-psychological and intrapersonal-cognitive of the questionnaire proposed in our study, but other specific reasons have not been consistent with our instrument.

Muller et al. (2015) conducted a study to design an instrument for identifying medication non-adherence factors. Three categories of barriers and reasons were identified in this study. The first group refers to intentional non-adherence, indicating that patients consciously decide to deviate from the treatment plan due to their attitudes or negative beliefs. The second category was unintentional non-adherence, like forgetfulness, depression, or lack of knowledge about the importance of adherence. The third category was factors related to medicines and the healthcare system, affecting non-adherence with medication [57]. Some intentional or unintentional obstacles are consistent with the intrapersonal-psychological and intrapersonal-cognitive dimensions. Furthermore, the obstacles related to medicines and the healthcare system are limitedly consistent with provider-related reasons in the present study. However, there is no item for identifying the socioeconomic reasons for non-adherence.

Most instruments measuring non-adherence with therapeutic recommendations are restricted to specific patient groups or do not cover all the reasons for non-adherence. Studies have evaluated non-adherence in patients with schizophrenia [19], hypertension [20], HIV [21], and tuberculosis [58], but the instrument provided in this study can be used for all patients in outpatient clinics.

Weinman et al. (2018) carried out a study to validate a valid instrument for non-adherence. This measurement instrument was limited to the reasons for the intentional non-adherence. For validity and evaluation of the instrument’s factorial structure, patients were selected from three clinical groups, including hypertension, oncology, and gout [24]. However, the present study questionnaire encompasses intentional and unintentional non-adherence with therapeutic recommendations prescribed by physicians. This point may be usefully applied to measure the non-adherence with medical recommendations for all diseases and at all outpatient settings in a general way.

Morisky et al. (2008) Evaluated the validity of an instrument of non-adherence in outpatient settings. This instrument is only applicable to medication non-adherence and in patients with hypertension. Items for this instrument assessed various psychosocial reasons for non-adherence and social support, satisfaction with care, and complexity of the medical regimen [59].

The questionnaire provided in this study measures all areas of treatment (medicine, visit, diagnostic services, laboratory services, non-medication recommendations such as diet and rehabilitation services and physiotherapy), while most of the previous instruments significantly evaluated the area of medicine [60, 61]. Morisky et al. questionnaire was designed to evaluate medication adherence [62]. Ghada Asaad et al. (2015) reviewed the validity of an instrument measuring dietary adherence in patients with type 2 diabetes [63].

As expected from relevant studies [37,38,39,40], the results of “known group comparisons” showed that females and patients with insurance had better adherence than male patients and those who lacked insurance.

Study limitations and strengths

One of the limitations of this study is that the questionnaire is limited to outpatient clinics and is designed to assess the reasons for non-adherence with medical orders in these settings and cannot be used in other health service providers and clinical environments such as hospitals. Also, according to the different contexts in countries, results may not be generalized to other countries. Another limitation is that one dimension’s reliability is less than the acceptable level, and further studies are needed to modify the items. Another limitation of this study is excluding some reasons, such as waiting time for visit, severity, and duration of diseases, lack of desired outcomes/meeting patients’ expectations, payment system, undesirable medication dosage prescribed by the physician, the cost-effectiveness of the visit and its effect on the frequent referral to different physicians. It should be noted that the experts excluded these factors as some of them were considered unnecessary. Some other reasons were excluded because the experts believed that patients have no knowledge and awareness about the health system’s features.

Because the questionnaire was designed based on the patients’ perspective, there are many essential limitations on conducting surveys on this group that should be considered, including the short form of the questionnaire, questionnaire completion time, and respondents’ patience in crowded healthcare centers increase its applicability. Long-form questionnaires are one reason for biased responses and can produce low-quality data [64, 65]. Also, it makes respondents bored and results in incorrect responses or less response rate [66]. Therefore, it can reduce the explanatory variance to a lower extent, which needs further research to design a longer form of a questionnaire with more components in other settings that can be more comprehensive. Last but not least, another limitation is, we did use no theoretical base of behavior change models, considering this issue that there are many predictors of non-adherence [67]. It is suggested that later research be focused on this issue.

Some reasons for non-adherence with medical orders may differ for different diseases and conditions [20, 58, 63]. However, the present study’s key strength is that the questionnaire identifies general reasons for non-adherence with therapeutic recommendations from patients’ perspectives in outpatient clinics; since there was no questionnaire to identify patients’ general behavior regardless of the type of disease and prescription in the outpatient clinics.

The present questionnaire can be used in outpatient clinics, and it is a recommendation for future studies, a questionnaire developing for non-adherence in inpatient settings. The questionnaire was also designed based on the patients’ perspectives, which recommendations for future studies to a questionnaire developing for determining the reasons for non-adherence from the providers’ perspectives.


According to the results, the questionnaire with four dimensions derived from exploratory factor analysis includes all dimensions of the perceived reasons for patients’ non-adherence with therapeutic recommendations. It is a valid and reliable instrument for identifying the perceived reasons for non-adherence with medical orders. Most of the similar instruments have measured medication non-adherence or have been limited to certain diseases. However, patients’ treatment is not solely based on medication non-adherence. This questionnaire is not limited to a specific area of non-adherence. It includes medical orders such as medicine-related recommendations, diagnostic services, and laboratory non-medication recommendations such as diet, physiotherapy, and rehabilitation services. This questionnaire can help policymakers identify the general reasons for non-adherence from patients’ perspectives in providing outpatient services, and as a result, to adopt tailored and effective policies and interventions to reduce non-adherence with adequate health care utilization and continuity of care. The questionnaire has good psychometric properties and can be used to investigate the reasons for non-adherence with therapeutic recommendations in outpatient clinics.

Availability of data and materials

The datasets analyzed during the current study are available from the corresponding author on reasonable request.



Exploratory Factor Analysis


Principal Component Analysis


  1. 1.

    Martin LR, Williams SL, Haskard KB, DiMatteo MR. The challenge of patient adherence. Ther Clin Risk Manag. 2005;1(3):189.

    PubMed  PubMed Central  Google Scholar 

  2. 2.

    Shay LE. A concept analysis: adherence and weight loss. Nurs Forum. 2008;43(1):42–52.

    PubMed  Article  PubMed Central  Google Scholar 

  3. 3.

    WHO. Adherence to long-term therapies: evidence for action. Geneva: World Health Organization; 2003.

    Google Scholar 

  4. 4.

    Nemes MIB, Helena ETS, Caraciolo JM, Basso CR. Assessing patient adherence to chronic diseases treatment: differentiating between epidemiological and clinical approaches. Cad Saude Publica. 2009;25:S392–400.

    PubMed  Article  PubMed Central  Google Scholar 

  5. 5.

    Boulet L-P, Vervloet D, Magar Y, Foster JM. Adherence: the goal to control asthma. Clin Chest Med. 2012;33(3):405–17.

    PubMed  Article  PubMed Central  Google Scholar 

  6. 6.

    Hsieh K-P, Chen L-C, Cheung K-L, Chang C-S, Yang Y-H. Interruption and non-adherence to long-term adjuvant hormone therapy is associated with adverse survival outcome of breast cancer women-an Asian population-based study. PLoS One. 2014;9(2):e87027.

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  7. 7.

    Vermeire E, Hearnshaw H, Van Royen P, Denekens J. Patient adherence to treatment: three decades of research. A comprehensive review. J Clin Pharm Ther. 2001;26(5):331–42.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  8. 8.

    Rai S, Mahapatra B, Sircar S, Raj PY, Venkatesh S, Shaukat M, et al. Adherence to antiretroviral therapy and its effect on survival of HIV-infected individuals in Jharkhand, India. PLoS One. 2013;8(6):e66860.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  9. 9.

    Bond W, Hussar D. Detection methods and strategies for improving medication adherence. Am J Health Syst Pharm. 1991;48(9):1978–88.

    CAS  Article  Google Scholar 

  10. 10.

    Cutler RL, Fernandez-Llimos F, Frommer M, Benrimoj C, Garcia-Cardenas V. Economic impact of medication non-adherence by disease groups: a systematic review. BMJ Open. 2018;8(1):e016982.

    PubMed  PubMed Central  Article  Google Scholar 

  11. 11.

    Sokol MC, McGuigan KA, Verbrugge RR, et al. Impact of medication adherence on hospitalization risk and healthcare cost. Med Care. 2005;43:521–30.

    PubMed  Article  PubMed Central  Google Scholar 

  12. 12.

    Ascher-Svanum H, Zhu B, Faries DE, Furiak NM, Montgomery W. Medication adherence levels and differential use of mental-health services in the treatment of schizophrenia. BMC Res Notes. 2009;2(1):6.

    PubMed  PubMed Central  Article  Google Scholar 

  13. 13.

    Wellard S. Constructions of chronic illness. Int J Nurs Stud. 1998;35(12):49–55.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  14. 14.

    Pang SK, Ip WY, Chang AM. Psychosocial correlates of fluid compliance among Chinese haemodialysis patients. J Adv Nurs. 2001;35(5):691–8.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  15. 15.

    Svarstad BL, Shireman TI, Sweeney J. Using drug claims data to assess the relationship of medication adherence with hospitalization and costs. Psychiatr Serv. 2001;52(6):805–11.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  16. 16.

    Loghman-Adham M. Medication non compliance in patients with chronic disease: issues in dialysis and renal transplantation. Am J Manag Care. 2003;9(2):155–71.

    PubMed  PubMed Central  Google Scholar 

  17. 17.

    Bissonnette J. Adherence: a concept analysis. J Adv Nurs. 2008;63:634–43.

    PubMed  Article  PubMed Central  Google Scholar 

  18. 18.

    Jin J, Sklar GE, Oh VMS, Li SC. Factors affecting therapeutic compliance: a review from the patient’s perspective. Ther Clin Risk Manag. 2008;4(1):269.

    PubMed  PubMed Central  Google Scholar 

  19. 19.

    Owie GO, Olotu SO, James BO. Reliability and validity of the medication adherence rating scale in a cohort of patients with schizophrenia from Nigeria. Trends Psychiatry Psychother. 2018;40(2):85–92.

    PubMed  Article  PubMed Central  Google Scholar 

  20. 20.

    Koschack J, Marx G, Schnakenberg J, Kochen MM, Himmel W. Comparison of two self-rating instruments for medication adherence assessment in hypertension revealed insufficient psychometric properties. J Clin Epidemiol. 2010;63:299–306.

    PubMed  Article  PubMed Central  Google Scholar 

  21. 21.

    Rathbun RC, Farmer KC, Lockhart SM, Stephens JR. Validity of a stage of change instrument in assessing medication adherence in indigent patients with HIV infection. Ann Pharmacother. 2007;41(2):208–15.

    PubMed  Article  PubMed Central  Google Scholar 

  22. 22.

    Sidorkiewicz S, Tran V-T, Cousyn C, Perrodeau E, Ravaud P. Development and validation of an instrument to assess treatment adherence for each individual drug taken by a patient. BMJ Open. 2016;6(5):e010510.

    PubMed  PubMed Central  Article  Google Scholar 

  23. 23.

    Naqvi AA, Hassali MA, Rizvi M, Zehra A, Iffat W, Haseeb A, et al. Development and validation of a novel general medication adherence scale (GMAS) for chronic illness patients in Pakistan. Front Pharmacol. 2018;9:1124.

    PubMed  PubMed Central  Article  Google Scholar 

  24. 24.

    Weinman J, Graham S, Canfield M, Kleinstäuber M, Perera AI, Dalbeth N, et al. The intentional non-adherence scale (INAS): initial development and validation. J Psychosom Res. 2018;115:110–6.

    PubMed  Article  PubMed Central  Google Scholar 

  25. 25.

    DeVon HA, Block ME, Moyle-Wright P, Ernst DM, Hayden SJ, Lazzara DJ, et al. A psychometric toolbox for testing validity and reliability. J Nurs Scholarsh. 2007;39(2):155–64.

    PubMed  Article  PubMed Central  Google Scholar 

  26. 26.

    Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35(6):382–5.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  27. 27.

    Beck CT, Gable RK. Ensuring content validity: an illustration of the process. J Nurs Meas. 2001;9(2):201–15.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  28. 28.

    Tilden VP, Nelson CA, May BA. Use of qualitative methods to enhance content validity. Nurs Res. 1990;39(3):172–5.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  29. 29.

    Naghavi S, Mehrolhassani MH, Nakhaee N, Yazdi-Feyzabadi V. Effective factors in non- compliance with therapeutic orders of specialists in outpatient clinics in Iran: a qualitative study. BMC Health Serv Res. 2019;19(1):413.

    PubMed  PubMed Central  Article  Google Scholar 

  30. 30.

    Mangione T. Mail surveys: improving the quality. Book 40 in applied social research methods. 1st ed. Thousand Oaks: SAGE Publications, Inc; 1995.

    Google Scholar 

  31. 31.

    Munro BH. Statistical methods for health care research. Philadelphia: Lippincott Williams & Wilkins; 2005.

    Google Scholar 

  32. 32.

    Lawshe CH. A quantitative approach to content validity. Pers Psychol. 1975;28(4):563–75.

    Article  Google Scholar 

  33. 33.

    Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30(4):459–67.

    PubMed  Article  PubMed Central  Google Scholar 

  34. 34.

    Drost EA. Validity and reliability in social science research. Educ Res Perspect. 2011;38(1):105–23.

    Google Scholar 

  35. 35.

    Ginty A. Construct validity. Encyclopedia of behavioral medicine. New York: Springer New York; 2013.

    Google Scholar 

  36. 36.

    Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best practices for developing and validating scales for health, social, and behavioral research: a primer. Front Public Health. 2018;6:149.

    PubMed  PubMed Central  Article  Google Scholar 

  37. 37.

    Naqvi AA, Hassali MA, Rizvi M, Zehra A. Validation of the general medication adherence scale in Pakistani patients with rheumatoid arthritis. Front Pharmacol. 2020;11:1039.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  38. 38.

    Kaplan RC, Bhalodkar NC, Brown EJ Jr, White J, Brown DL. Race, ethnicity, and sociocultural characteristics predict noncompliance with lipid-lowering medications. Prev Med. 2004;39(6):1249–55.

    PubMed  Article  PubMed Central  Google Scholar 

  39. 39.

    Lindberg M, Ekström T, Möller M, Ahlner J. Asthma care and factors affecting medication compliance: the patient's point of view. Int J Qual Health Care. 2001;13(5):375–83.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  40. 40.

    Hadji P, Jacob L, Kostev K. Gender-and age-related treatment compliance in patients with osteoporosis in Germany. Patient Preference Adherence. 2016;10:2379.

    PubMed  Article  PubMed Central  Google Scholar 

  41. 41.

    Henson RK, Roberts JK. Use of exploratory factor analysis in published research: common errors and some comment on improved practice. Educ Psychol Meas. 2006;66(3):393–416.

    Article  Google Scholar 

  42. 42.

    Nunnally J, Berstein I. Psychometric theory. 3rd ed. New York: McGraw-Hill; 1994. p. 248–70.

    Google Scholar 

  43. 43.

    Kaiser HF, Rice J. Little jiffy, mark IV. Educ Psychol Meas. 1974;34(1):111–7.

    Article  Google Scholar 

  44. 44.

    Kim J, Lee W, Hong Y, et al. Psychometric properties of a short self-reported measure of medication adherence among patients with hypertension treated in a busy clinic setting in Korea. J Epidemiol 2014;24:132–40.

  45. 45.

    Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. New York: Oxford University Press; 2008.

    Google Scholar 

  46. 46.

    Atinga RA, Yarney L, Gavu NM. Factors influencing long-term medication non-adherence among diabetes and hypertensive patients in Ghana: a qualitative investigation. PLoS One. 2018;13(3):e0193995.

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  47. 47.

    Orom H, Underwood W III, Cheng Z, Homish DL, Scott IY. Relationships as medicine: quality of the physician–patient relationship determines physician influence on treatment recommendation adherence. Health Serv Res. 2018;53(1):580–96.

    PubMed  Article  PubMed Central  Google Scholar 

  48. 48.

    Phillips KA, Kerlikowske K, Baker LC, Chang SW, Brown ML. Factors associated with women's adherence to mammography screening guidelines. Health Serv Res. 1998;33(1):29.

    CAS  PubMed  PubMed Central  Google Scholar 

  49. 49.

    Nguyen GC, LaVeist TA, Harris ML, Datta LW, Bayless TM, Brant SR. Patient trust-in-physician and race are predictors of adherence to medical management in inflammatory bowel disease. Inflamm Bowel Dis. 2009;15(8):1233–9.

    PubMed  PubMed Central  Article  Google Scholar 

  50. 50.

    Goodhand J, Kamperidis N, Sirwan B, Macken L, Tshuma N, Koodun Y, et al. Factors associated with thiopurine non-adherence in patients with inflammatory bowel disease. Aliment Pharmacol Ther. 2013;38(9):1097–108.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  51. 51.

    Makanjuola T, Taddese HB, Booth A. Factors associated with adherence to treatment with isoniazid for the prevention of tuberculosis amongst people living with HIV/AIDS: a systematic review of qualitative data. PLoS One. 2014;9(2):e87166.

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  52. 52.

    Zegeye A, Dessie G, Wagnew F, Gebrie A, Islam SMS, Tesfaye B, et al. Prevalence and determinants of anti-tuberculosis treatment non-adherence in Ethiopia: a systematic review and meta-analysis. PLoS One. 2019;14(1):e0210422.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  53. 53.

    Semvua SK, Orrell C, Mmbaga BT, Semvua HH, Bartlett JA, Boulle AA. Predictors of non-adherence to antiretroviral therapy among HIV infected patients in northern Tanzania. PLoS One. 2017;12(12):e0189460.

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  54. 54.

    Ghimire S, Castelino RL, Jose MD, Zaidi STR. Medication adherence perspectives in haemodialysis patients: a qualitative study. BMC Nephrol. 2017;18(1):167.

    PubMed  PubMed Central  Article  Google Scholar 

  55. 55.

    Chizzola PR, Mansur AJ, Luz PL, Bellotti G. Compliance with pharmacological treatment in outpatients from a Brazilian cardiology referral center. Sao Paulo Med J. 1996;114(5):1259–64.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  56. 56.

    Jank S, Bertsche T, Schellberg D, Herzog W, Haefeli WE. The A14-scale: development and evaluation of a questionnaire for assessment of adherence and individual barriers. Pharm World Sci. 2009;31:426–31.

    PubMed  Article  PubMed Central  Google Scholar 

  57. 57.

    Muller S, Kohlmann T, Wilke T. Validation of the adherence barriers questionnaire–an instrument for identifying potential risk factors associated with medication-related non-adherence. BMC Health Serv Res. 2015;15(1):153.

    PubMed  PubMed Central  Article  Google Scholar 

  58. 58.

    Yin X, Tu X, Tong Y, Yang R, Wang Y, Cao S, et al. Development and validation of a tuberculosis medication adherence scale. PLoS One. 2012;7(12):e50328.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  59. 59.

    Morisky DE, Ang A, Krousel-Wood M, Ward HJ. Predictive validity of a medication adherence measure in an outpatient setting. J Clin Hypertens. 2008;10(5):348–54.

    Article  Google Scholar 

  60. 60.

    Wetzels G, Nelemans P, van Wijk B, Broers N, Schouten J, Prins M. Determinants of poor adherence in hypertensive patients: development and validation of the “Maastricht Utrecht adherence in hypertension (MUAH)-questionnaire”. Patient EducCouns. 2006;64:151–8.

    Article  Google Scholar 

  61. 61.

    Holstad MM, Foster V, DiIorio C, McCarty F, Teplinskiy I. An examination of the psychometric properties of the antiretroviral general adherence scale (AGAS) in two samples of HIV-infected individuals. J Assoc Nurses AIDS Care. 2010;21:162–72.

    PubMed  Article  PubMed Central  Google Scholar 

  62. 62.

    Morisky DE, Green LW, Levine DM. Concurrent and predictive validity of a self-reported measure of medication adherence. Med Care. 1986;24:67–74.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  63. 63.

    Asaad G, Sadegian M, Lau R, Xu Y, Soria-Contreras DC, Bell RC, et al. The reliability and validity of the perceived dietary adherence questionnaire for people with type 2 diabetes. Nutrients. 2015;7(7):5484–96.

    PubMed  PubMed Central  Article  Google Scholar 

  64. 64.

    Galesic M, Bosnjak M. Effects of questionnaire length on participation and indicators of response quality in a web survey. Public Opin Q. 2009;73(2):349–60.

    Article  Google Scholar 

  65. 65.

    Crawford SD, Couper MP, Lamias MJ. Web surveys perceptions of burden. Soc Sci Comput Rev. 2001;19(2):146–62.

    Article  Google Scholar 

  66. 66.

    Nakash RA, Hutton JL, Jørstad-Stein EC, Gates S, Lamb SE. Maximising response to postal questionnaires – A systematic review of randomised trials in health research. BMC Med Res Methodol. 2006;6(1):1–9.

  67. 67.

    Vrijens B, De Geest S, Hughes DA, Przemyslaw K, Demonceau J, Ruppar T, et al. A new taxonomy for describing and defining adherence to medications. Br J Clin Pharmacol. 2012;73(5):691–705.

    PubMed  PubMed Central  Article  Google Scholar 

Download references


The present study is based on the master’s degree dissertation in the field of health services management. The authors would like to gratitude all the participants in the present study.


This study is part of a MSc dissertation, financed by Research and Technology Deputy affiliated with the Kerman University of Medical Sciences (KUMS) with grant no 96001126. The funding body was not involved in the study design, study execution, or the writing of this manuscript. We also thank the Health Services Management Research Center affiliated with the Institute for Futures Studies in Health at KUMS for the financial support and facilitating this research.

Author information




VYF designed the study, supervised its implementation, assisted in analyzing the data, and assisted in preparing the manuscript. NN helped analyze the data, provided some guidance in the study’s implementation, and assisted in preparing the manuscript. MHM provided mentoring in the study’s implementation, assisted in Formal analysis, and prepared the manuscript. SN conducted the Data curation, analyzed the data, and led the preparation of the manuscript. EHR assisted in the preparation of the manuscript. All authors read and approved the final version of the article.

Corresponding author

Correspondence to Soheila Naghavi.

Ethics declarations

Ethics approval and consent to participate

The study got approved by the Ethical Committee of Kerman University of Medical Sciences (Approval ID: IR.KMU.REC.1397.040). Ethical considerations were taken into account during data collection and analysis. Any question asked about the study by the participant was explained in detail to eliminate the potential ambiguities. Furthermore, the written informed consent was obtained from the participants before filling out the questionnaire. It was assured that confidentiality would be maintained and that no personal information would be identified in any publications arising from the study. Participants were informed that participation was entirely voluntary, and they could withdraw at any time.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests. Dr. Vahid Yazdi-Feyzabadi is an Editorial Board member.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Final version of the developed questionnaire.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yazdi-Feyzabadi, V., Nakhaee, N., Mehrolhassani, M.H. et al. Development and validation of a questionnaire to determine medical orders non-adherence: a sequential exploratory mixed-method study. BMC Health Serv Res 21, 136 (2021).

Download citation


  • Non-adherence
  • Prescriptions
  • Ambulatory care facilities
  • Instrument development
  • Psychometric evaluation