Document Type : Original article
Subjects
Table 1. Content validity form for developing the quality of PHC services questionnaire
|
Score |
CVI |
CVR |
||
|
Transparency |
Relevance |
Simplicity |
Necessity |
|
|
4 |
Completely transparent |
Completely relevant |
Completely simple |
Necessary |
|
3 |
Transparent, but need minor changes |
Relevant, but need minor changes |
Simple, but need minor changes |
Useful but not necessary |
|
2 |
Need some changes |
Need some changes |
Need some changes |
Not necessary |
|
1 |
Not transparent |
Irrelevant |
Not simple |
|
Reliability
In the present study, to determine the reliability and the stability of responses, the questionnaire and the internal consistency of the items and scales were evaluated. The test-retest method was used to determine the stability of the responses, and Cronbach’s alpha coefficient was calculated to evaluate and determine the internal consistency of the questionnaire. In the first approach, the respondents completed the questionnaire twice with a two-week interval, and the correlation coefficient between the scores was calculated. In the second approach, the internal consistency of the items and scales was evaluated by determining the Cronbach’s alpha coefficient for the entire questionnaire and for each of its scales independently.
In order to determine the reliability of the questionnaire by Cronbach’s method, 650 service recipients were recruited, and to check the reliability by the before-and-after method, 30 individuals answered the questions again after two weeks.
One of the tests that should be done to check the reliability in clinical environments, which is objective, reliable and valid, is the test-retest (26).
Confirmatory Factor Analysis
Confirmatory Factor Analysis (CFA) is interconnected statistical techniques. When some concepts relation is to be tested, the researcher uses CFA to test a hypothetical model of the system he/she is trying to propose. Thus, CFA helps in identifying the factor structure we believe the phenomena follows or is described by (27).
To evaluate the factor construct, the confirmatory factor analysis model was used. Data were analyzed using IBM statistic SPSS version 23 and IBM SPSS Amos 22.
Table 2. Demographic characteristics of the participants in the primary health service quality question nairetest
|
Variable |
Number (percent) |
Mea (standard deviation)
|
Test statistics |
|
|
Gender |
Male |
73(11.2%) |
1.96(0.43%) |
t=-1.131 P=0.259 |
|
Female |
577(88.8%) |
2.02(0.40%) |
||
|
Age (year) |
<20 |
23(3.5%) |
1.66(0.47%) |
F=16.624
P<0.001 |
|
20-30 |
199(30.6%) |
1.87(0.47%) |
||
|
31-40 |
286(44%) |
2.09(0.33%) |
||
|
41-50 |
126(19.4%) |
2.10(0.34%) |
||
|
>50 |
16(2.5%) |
2.20(0.33%) |
||
|
Education |
No degree |
232(35.7%) |
2.03(0.41%) |
F=4.717
P<0.001 |
|
High school degree |
165(25.4%) |
1.90(0.41%) |
||
|
A.D |
169(26%) |
2.04(0.33%) |
||
|
B.S |
73(11.2%) |
2.10(0.42%) |
||
|
M.S |
11(1.7%) |
2.06(0.62%) |
||
Results
Characteristics of the participants
The study was conducted with 650 participants. Most of the participants were female (88.8%) and the education level under diploma (35.7%). Considering that most of the provided PHC services are related to children and locally it has been cultured that this responsibility is on mothers, most of the participants were female. Also, due to the fact that the working hours of the PHC centers are coincide with working hours of men, usually most of the SRs are women.
Most of the participants were from the age group of 31 to 40 years old (44%). In terms of income level, most of the participants (51.4%) had an income of 300 to 600 dollars per month. The relationship between the participants’ gender and their perception of service quality was not significant. On the other hand, the correlation between the participants’ age, education and income with their perception of the quality of PHC services was significant. In this way, with the increase in age, education and income, the quality of services was reported higher (Table 2).
Reliability
In order to check the standard of the designed questionnaire, first the validity and reliability of the extracted questionnaire was evaluated. To check the reliability, through internal consistency Cronbach’s alpha coefficient and test-retest method, Pearson’s coefficient were calculated. The examination of the internal consistency of the questionnaire constructs showed that all the scales and subscales of the questionnaire have the minimum reliability standard (0.7), and according to the Cronbach’s alpha coefficients, the internal consistency of all constructs of the PHC service quality questionnaire is acceptable (Table 3).
Content validity
In order to check the standard of the designed questionnaire, first the validity and reliability of the extracted questionnaire was evaluated. The content validity form was provided to 14 faculty members and 12 experts with executive experience in PHC, and they rated each item.
For qualitative content validity, based on experts’ opinions, 3 components out of 13 components were removed and 5 components were integrated into other components. Through the quantitative content validity assessment, content validity indices were calculated for all the 104 extracted items, as a result of which 48 items were removed due to CVR<0.42 and CVI<0.79. Finally, 46 items in the form of 5 components were obtained.
Normality test
After confirming the standard, the initial questionnaire was distributed to the sample size (n-650). One of the main assumptions of statistical tests is to check the normality of the research variables distribution. Checking the normality of the data has been evaluated through the skewness and kurtosis index in SPSS software. If the skewness and kurtosis are between -2 and +2, the data distribution is normal (Table 4).
Considering that the value of skewness and kurtosis was obtained between -2 and +2, the parametric tests are used in the following.
Table 3. The results of the evaluation of internal consistency (Cronbach’s α) and test-retest of the quality ofprimary health services questionnaire
|
Quality dimensions |
Sample Size |
Cronbach’s α |
Test-retest (ICC) |
|
Patient-centeredness |
30 |
0.83 |
0.724 |
|
Continuity of care |
30 |
0.778 |
0.772 |
|
Quality of basic amenities |
30 |
0.806 |
0.861 |
|
Access and availabiliry |
30 |
0.751 |
0.786 |
|
Trust |
30 |
0.742 |
0.854 |
|
Total quality |
30 |
0.942 |
0.868 |
Table 4. Skewness and kurtosis test results
|
Research variables |
Sample Size |
Kurtosis |
Skewness |
Result |
Test type |
|
Patient-centeredness |
650 |
0.652 |
1.313 |
Confirm |
Parametric |
|
Continuity of care |
650 |
-0.410 |
1.087 |
Confirm |
Parametric |
|
Quality of basic amenities |
650 |
1.120 |
0.224 |
Confirm |
Parametric |
|
Access and availability |
650 |
0.325 |
.500 |
Confirm |
Parametric |
|
Trust |
650 |
0.871 |
.263 |
Confirm |
Parametric |
Table 5. Fit indices of the five-factor model for measuring the quality of primary health services (n=650)
|
Model |
χ2 |
df |
χ2/df |
RMSR |
RMSEA (90% CI) |
CFI |
GFI |
AGFI |
|
Total quality |
3251.041 |
972 |
3.34 |
0.038 |
0.060 (0.058;0.062) |
0.783 |
0.777 |
0.752 |
Construct validity
Since in exploratory factor analysis, all the items were classified as one factor, and there was no scientific and logical reason to follow the results of exploratory factor analysis, the items were distributed among different factors based on the opinions of experts, and confirmatory factor analysis was done on the factors determined by the experts.
The results of the confirmatory factor analysis confirmed the five-factor model, determined by the experts, in terms of the confirmation of the model by the fit indices (Table 5, Figure 2).
Final questionnaire and scoring method
The final scale for evaluating the service quality in PHC was finally extracted and presented in the form of five main dimensions and 46 items (Table 6).
Table 6. Final designed questionnaire
|
Criteria |
Sub-criteria |
|
Patient-centeredness |
1. During the service delivery process, do the staffs patiently listen to you and answer all your questions in a friendly manner? |
|
2. During the service delivery process, do the staffs encourage you to express your feelings and concerns about your health problems or needs? |
|
|
3. Is the presence and participation of other family members, especially men, welcomed in the center? |
|
|
4. If anyone is willing, are people’s participation in health matters welcomed? |
|
|
5. Do the staffs of this center ask for your opinion and involve you in making decisions about your own health matters? |
|
|
6. Are you allowed to choose or reject health services (without pressure and coercion from staffs)? |
|
|
7. Do the staff talk to you in such a way that other people cannot hear you? |
|
|
8. If an examination is needed, do the personnel do it out of the other’s sight? |
|
|
9. Do the personnel respect your beliefs, values and convictions? |
|
|
10. Have you ever felt that due to things such as gender, lack of money, ethnicity, socio-economic status, etc., proper services were not provided to you? |
|
|
11. Are special services (such as injections) provided separately by male staff for men and female staff for women? |
|
|
Continuity of care |
12. Are you satisfied with the way of following up the referral result after referral to diagnostic and clinical centers? |
|
13. Will the center follow up if you do not refer to PHC center on time? (Through phone calls, door-to-door visits, or follow-up through health volenteers) |
|
|
14. Is it possible to provide services permanently by one of the staffs in your usual visits? |
|
|
15. If one of the staffs is not present, will other personnel provide your required services? Or will the necessary information be provided to you? |
|
|
16. Are the days when a certain service is not provided informed? |
|
|
17. If one of the personnel be absent, will the other personnel do his\her tasks? |
|
|
18. If the center does not provide a certain service at time, will other centers (other than the center in which you registered) provide the service to you? |
|
|
19. In case of referral to other health centers, is there coordination with the destination center? (introduction letter, phone call, etc.) |
|
|
20. After you are referred to a specialist or to diagnostic and clinical centers, is there a follow-up to achieve results? |
|
|
21. If you make a phone call, do the staffs answer your questions properly? |
|
|
Basic amenities |
22. Are the personnel of the center dressed properly and do they consider personal hygiene? |
|
23. Is there enough space and chairs to sit in the waiting room\salon? |
|
|
24. Is the location of the service departments suitable and is it easy to move between the departments? (No stairs, large distance between departments, lack of physical obstacles in the place of movement between departments, etc.) |
|
|
25. Is there a separate room for each service provider department? |
|
|
26. Is the space allocated to each room appropriate? |
|
|
27. Are there chairs inside the rooms to sit while receiving service and talking to the staffs? |
|
|
28. Is the environment of the center fun and lively for children and other people? |
|
|
29. Is the center or department properly cleaned? |
|
|
30. Is the heating and cooling system of the center suitable in the cold and hot seasons of the year? |
|
|
31. Is there drinking water for service recepients? |
|
|
32. Are there clean and suitable washrooms and toilets for service recepients? |
|
|
33. Are there floor or room guides installed sufficiently in appropriate places? |
|
|
Access and availability |
34. Is there an appointment system (based on day and time) in this center? |
|
35. Is the waiting time, to receive service, appropriate? |
|
|
36. Is the waiting time, to receive emergency service, appropriate? |
|
|
37. Has been ever the long distance from your home to the health center led you to refuse to receive a service? |
|
|
38. Are the working hours of this center proper for you? |
|
|
39. Do you get services at any time you refer during the working hours of PHC center? |
|
|
40. Are there health educational references (books, booklets, pamphlets, etc.) available to you when visiting the PHC center? |
|
|
Trust |
41. Do the staffs understand and diagnose your problem well? |
|
42. Do you follow the orders and instructions of the staffs? |
|
|
43. Have the services provided in this center been effective in solving the problem or responding to your health needs? |
|
|
44. Do the services provided in this center cover all your health needs? |
|
|
45. Are you confident about the secrecy of the staffs and the confidentiality of the information you give to the service providers or the contents of your household health file? |
|
|
46. When providing services and in training classes, do the personnel present the contents in such a way that you can easily deal with some of your problems in the field of the taught topic? |
To score the items of the designed questionnaire and obtain an overall picture of service quality, a method invented by a health service quality institute in the Netherlands, was used (28). In this way, the designed questionnaire’ items were scored from the two aspects of importance and performance. The respondents were asked to score the importance of each item and their perception of the quality of care they received in relation to that item (performance). The importance of service quality was scored on a four-point Likert scale from 1 to 10: 0=not important, 3=moderately important, 6=important, and 10=very important.
On the other hand, the perceived performance of the received care was scored on a four-point scale of “never, sometimes, often, and always.”
For analysis, this scale was dichotomized: 0=often/always and 1=never/sometimes.
Finally, the quality of care services for each quality aspect was calculated as follows:
Service quality=10 - (importance x performance).
Then, the service quality score ranged from 0=worst/lowest quality to 10=best/highest quality.
According to experts, a score of less than 5 indicates poor quality, between 5 and 7.5 average, and more than 7.5 demonstrates good service quality.
On the other hand, in most surveys, regardless of the methodology, about 10% of the population reported inadequate quality of care, and a similar percentage were dissatisfied with care in hospitals. Given this result, it is logical that a service quality score of less than 9 represents a significant opportunity for improvement (29).
Discussion
Service quality assessment is a valuable approach to measure the performance of the health system in providing PHC and to identify its strengths and weaknesses and it can provide important information for designing effective strategies for health care management. Therefore, the aim of this study was to develop and validate a patient-centered questionnaire to measure the quality of PHC services.
Based on the results of the study, after measuring the validity and reliability of the primary items, the PHC quality assessment questionnaire was finalized with 46 questions and five conceptual dimentions including “Patient-centeredness”, “Continuity of care”, “Quality of basic amenities”, “Access and availabiliry” and “Trust”. In the study of He et al in China, a questionnaire was designed to measure the quality of PHC services, which has three dimensions: service attitude (patient participation in decision-making, patient-physician relationship and staff behavior), facilities and staff skills (hospital environment, hospital facilities and skill of physicians and nurses) and the patients cost (waiting time, distance to the hospital and cost of services) (30). The subscales and items extracted in the present study cover all the dimensions obtained in the mentioned study. The only difference lies in the cost to patients, which may result from the fact that PHC services in Iran are provided free of charge.
Also, in the study of Gage et al in Haiti, the quality of PHC services were measured using two scales “service readiness index” (including the five dimensions of essential drugs; diagnostic power; basic equipment; infection prevention; and basic facilities) and “PHC Performance Initiative (PHCPI)” (including the four dimensions of access to services; effective service delivery (trust and belief in the competence of staff); managing and organizing; and patient-centeredness) (31).
Some dimensions defined for PHC quality in the study of Gage et al (31) including essential drugs, diagnostic power, managing and organizing and infection prevention are not emerged in the present study. Various reasons for this difference can be stated, including the lack of diagnostic services and limited provision of pharmaceutical services in PHC in Iran. Additionally, one of the possible reasons for the absence of the dimension of managing and organizing may be that people perceive the output of managing and organizing in service quality. Also, probably due to the lack of services such as limited surgery, injections, dressings and similar interventions in PHC in Iran, infection prevention has not been the focus of SRs in Iran.
On the other hand, in the current study, one of the dimensions defined for the quality of PHC is continuity of care, which was not considered in the study of Gage et al. One of the possible reasons for this issue can be the lack of integration of PHC and higher levels of service provision and the absence of mandatory referral system in Iran.
In the study of Kaitelidou et al in Greece, the dimensions of the quality of PHC services included access, technical quality of services, interpersonal aspects (comprising trust in the ability of personnel and patient-centered), continuity of cares and physical environment (including the space for providing services such as the cleanliness, the convenience of service recipients, room signs, etc.) (32). As it can be comprehended, all the dimensions identified in the study of Kaitelidou et al are covered in the present study. The similarity between the quality dimensions of PHC services in different countries and populations indicates that the designed questionnaire in this study can be a basis for conducting similar studies in other countries.
On the other hand, in different countries with different health systems, PHC is provided with different structures and methods, which can affect the definition and the dimensions of the quality of PHC services. In the systematic review study by Dorri Kafrani et al which investigated the patterns of measuring and improving quality in PHC worldwide, 8 different models for evaluating and improving the quality of PHC have been presented, each of which can be used individually or in combination for different contexts according to the limitations and facilities (33). Therefore, in the present study, it has been tried to develop and validate a questionnaire based on the specific conditions and characteristics of PHC in Iran, in order to provide the possibility of identifying and solving challenges related to quality of services.
With the aim of finding a suitable tool for measuring the quality of healthcare services that is in accordance with the unique nature of the health system in different countries, Endeshaw examined and compared different models of health service quality assessment from different dimensions in the form of a literature review study (10). The author of the study finally concluded that the existing western frameworks and tools are incompatible with the health context of developing countries due to cultural and economic differences. In the literature review, it was confirmed that each country and even each healthcare service organization should have its own framework for measuring the quality of healthcare services. In addition, a continuous effort should be made to redefine quality criteria and study the complex issues of service quality in the health care setting.
In this study, the face, content and construct validity of the designed questionnaire were evaluated. Based on the results of qualitative and quantitative face validity, in the view of the participant, all the items of the questionnaire had good fit and connection with its main purpose and were understandable. Also, the qualitative and quantitative content validity confirmed the necessity of the 46 items and the simplicity, relevance and clarity of each of them according to the experts. The results of confirmatory factor analysis also confirmed the extracted model.
In the present study, the reliability assessment of the PHC service quality questionnaire showed that all the scales and subscales of the questionnaire have good internal consistency. Also, the reliability of the questionnaire was assessed using the test-retest method on 30 participants at a two-week interval. Examining the reliability of the questionnaire using the test-retest method is one of the advantages of the present study compared to other studies that only use Cronbach’s alpha method (30,32,34-36).
The results of study revealed that there is no difference between women and men in terms of people’s perception of the quality of PHC services. This result can indicate gender justice in providing services to the society. The study by Rezaei et al (37) which was conducted in Iran with the aim of comparing the services quality in urban and rural PHC centers and the study by Chung et al (38) in China and Dullie et al (35) study in Mali also obtained the same result. The results of these studies suggested that there is no significant difference between the gender of people and their perception of service quality. The findings of the present study are in contrast with the findings of the study of Bustamante et al, who showed that women’s perception of the quality of PHC services is different from men (39).
On the other hand, income, the level of education and age of individuals affect their perception of the quality of PHC services. In the studies conducted in Iran and other countries, there was no significant relationship between people’s age and level of education with their perception of the quality of PHC services (34,36,37). The reason for this difference can be the use of different measures or can be the different structures of providing PHC services. To find out the reason for this result, it is necessary to conduct more studies using developed questionnaire in different populations.
Study limitations
One of the limitations of the study is testing the developed questionnaire on the urban community in one province, which can make it difficult to generalize the results to rural communities and other provinces. Also, since the questionnaire designed in this study is a self-report measure, the answers may be affected by response bias. It is suggested that in future studies to more test the questionnaire, surveys should be conducted among different populations in order to obtain more reliable results.
Conclusion
In this study, a valid and reliable questionnaire was developed to measure the quality of PHC services from the perspective of service recipients. This tool can be useful in examining the difference between PHC centers and different populations, and provide the basis for improving the management and the quality of PHC services, thus helping to improve the individual and public health. Also, this measure can be used as a practical guide for future studies in order to investigate the possible relationships between independent variables (for instance, demographic variables) or the impact of service quality on health outcomes such as service utilization, service effectiveness, etc.
Acknowledgement
Initially, the authors would like to thank the participants in the study, who expressed their view about the quality of PHC services. They would also like to thank the officials of district health center of Hamadan city and the UPHCC staffs who helped in FGDs and other steps of the study.
This study was approved by the Ethics Committee of authors’ institute (ethical number: IR.UMSHA.REC.1395.453).
Conflict of Interest
There was no conflict of interest in this manuscript.
------WebKitFormBounda^ryuTq6kilIdufU5RNq Content-Disposition: form-data; name="dor"