Evaluation of Pediatrics Residency Training Program in Hormozgan University of Medical Sciences Using CIPP Evaluation Model in 2020

Document Type : Original article

Authors

1 Department of Medical Education, Shiraz University of Medical Sciences, Shiraz, Iran.

2 Department of Medical Education, Shiraz University of Medical Sciences, Shiraz, Iran

3 Clinical Education Research Center, Education Development Center, Shiraz University of Medical Sciences, Shiraz, Iran

Abstract

Abstract
Background: The curriculum is the most important component of the educational system and evaluation is one of the most significant and sensitive components of the curriculum too. Through evaluation, the efficiency and effectiveness of programs can be increased.
Methods: This is a descriptive cross-sectional study that was conducted to evaluate the residency curriculum of the Pediatrics Department of Hormozgan Medical School using the CIPP model. The statistical population in the study included 20 Pediatrics residents and 20 faculty members who were selected by the census method. Researcher-made questionnaires were used to collect data. Data analysis was performed via descriptive-analytical statistical methods using SPSS version 22.
Results: The dimensions of context, input, process, and product were evaluated as relatively desirable. The context, input, process, and product status had a statistically significant difference from the participant’s perspective. According to (p≤0.01) and the average score of faculty members and residents respectively (M=3.15) and (M=2.57), it can be said that faculty members have a more positive attitude towards residents in evaluating the Pediatrics residency curriculum except for the product dimension.
Conclusion: Generally the curriculum of the Pediatrics Department of Hormozgan Medical School was evaluated as relatively desirable from the perspective of faculty members and residents. This issue shows, there is a long way to improve the quality of the curriculum, which will be achieved by eliminating the shortcomings in various areas of education. Consequently, it will be effective to evaluate the periodical of the Pediatrics department curriculum to improve it and achieve long-term educational goals.

Keywords


Abstract
Background: The curriculum is the most important component of the educational system and evaluation is one of the most significant and sensitive components of the curriculum too. Through evaluation, the efficiency and effectiveness of programs can be increased.
Methods: This is a descriptive cross-sectional study that was conducted to evaluate the residency curriculum of the Pediatrics Department of Hormozgan Medical School using the CIPP model. The statistical population in the study included 20 Pediatrics residents and 20 faculty members who were selected by the census method. Researcher-made questionnaires were used to collect data. Data analysis was performed via descriptive-analytical statistical methods using SPSS version 22.
Results: The dimensions of context, input, process, and product were evaluated as relatively desirable. The context, input, process, and product status had a statistically significant difference from the participant’s perspective. According to (p≤0.01) and the average score of faculty members and residents respectively (M=3.15) and (M=2.57), it can be said that faculty members have a more positive attitude towards residents in evaluating the Pediatrics residency curriculum except for the product dimension.
Conclusion: Generally the curriculum of the Pediatrics Department of Hormozgan Medical School was evaluated as relatively desirable from the perspective of faculty members and residents. This issue shows, there is a long way to improve the quality of the curriculum, which will be achieved by eliminating the shortcomings in various areas of education. Consequently, it will be effective to evaluate the periodical of the Pediatrics department curriculum to improve it and achieve long-term educational goals.
Keywords: Censuses, Data analysis, Humans, Schools, Medical internship and residency, Universities

Introduction 
Program evaluation is one of the essential strategies for receiving feedback and can move education from steady to dynamic (1). Educational evaluation provides a mirror for decision-makers and those involved in educational activities to see how the activities work (2). Nowadays, in educational centers, due to its unique nature and function, evaluation is one of the most widespread and controversial educational topics used as a determining factor in the usefulness and effectiveness of educational programs in all prestigious higher education centers worldwide. Therefore, due to the need to train the experienced workforce to provide health services with the desired quality, evaluation programs in the medical education system are particularly important. Evaluation is the feedback of the steps of a process based on which the degree of achievement or non-achievement of goals can be explored (3). Educational program evaluation is a formal activity performed to determine the quality, effectiveness, value, process, goal, or curriculum (4).
There are different categories of evaluation models, one of which is the management-based model. The managerial evaluation model is also called the decision-making model since it helps decision-makers and managers decide educational activities (5). One of the types of models to facilitate decision-making is the Context, Input, Process, and Product (CIPP) model. Stufflebeam et al are the creators of this evaluation model (1,6). CIPP is derived from the first letters of the words Context, Input, Process, and Product. The CIPP model provides a comprehensive framework for evaluation in a variety of areas, including the curriculum. The most crucial purpose of evaluation in the CIPP model is to improve program performance. This pattern helps program staff regularly modify their program during and at the end of the program via collecting information (7).
The purpose of context assessment is to provide a rational and logical basis for setting educational goals. It is also the analytical effort to identify relevant elements in the learning environment and identify problems, needs, and opportunities in an educational context or situation. Input evaluation helps to design and select appropriate methods to achieve the goals (8). Process evaluation is performed to diagnose or predict organizational problems during educational activities and the desirability of the implementation process of these activities (9). Product evaluation is performed to judge the desirability of the efficiency of educational activities (10). The program results are compared with the program’s goals, and the relationship between expectations and actual results is determined. In evaluations, all individuals and departments affected by the training program are used as judges, and what affects the performance of the set stakeholders is evaluated. The results are presented to the program decision-makers as shown in the study of Bilan et al in which the highest degree of satisfaction was associated with the output indices and the lowest degree with input elements. Thus, it was concluded that the resources and device were not optimal (11-13).
Pediatrics medical education in Iran began for the first time at University of Tehran with a Pediatrics department by Dr. Mohammad Gharib. This department was the first educational department for Pediatrics in Iran’s universities, formed in 1941 and started its educational and research activities at the level of general medicine (14). Since the educational program is the most essential and fundamental component of the higher education system, it can be stated that the educational program and its dimensions reflect the level of progress and responsiveness of universities to the changing needs of societies. An educational program becomes effective when its performance is evaluated correctly (15) because educational evaluation is the best indicator that shows the level of achievement to educational goals and analyzes the educational activities (13,16). 
Monitoring and evaluation of the Pediatrics residency educational program have particular importance since residents deal with the health of Pediatrics from birth to adulthood (end of eighteen years). The educational program of the specialized field of Pediatrics includes health education, diagnosis, and treatment of various diseases of infants, Pediatrics, and adolescents. Since the progress and development of any country depends on the health of society, especially Pediatrics, Iranian Pediatrics, who make up about 40% of our society, can have a bright future if they have physical and mental health (17,18). Thus, to ensure the quality of the educational program for residents, the educational program must be continuously assessed. The method of implementation of the program, the capacity of faculty members, and their function as guides and effective organizing of the educators’ experiences are among the factors that affect the learning experiences and, ultimately, the quality of physicians and experts who receive medical licenses (19).
In the field of evaluation, using the CIPP model, research has been conducted outside and inside Iran. A study entitled “Using the CIPP model to assess nursing education program quality and merit” stated that the CIPP model provides a flexible framework and perspective for managing the details of curriculum evaluation. Managers and educators should consider the CIPP model reliable for evaluating educational programs’ quality, value, and improvement (20). A review of past studies shows that educational institutions have always faced evaluation as a challenging issue. Training center programs are effective when their performance is evaluated in different ways. The CIPP evaluation model improves the activities and success of programs due to the participation of active elements in the evaluation and motivation of the education system in the study of opportunities and challenges. This pattern also provides information that can be used to make wise decisions in context, input, process, and output. Higher education programs need quantitative and qualitative evaluation to improve quality and dynamism.
Hormozgan Medical School, since 1994, with the review and approval of the Ministry of Health and Medical Education, has accepted and trained assistants in the field of Pediatrics. The Pediatrics department, with 22 faculty members, trains an average of 23 specialized assistants annually in Hormozgan Pediatrics Hospital, with 152 active beds.
Accordingly, considering the lack of studies evaluating the Pediatrics residency program in Hormozgan University of Medical Sciences based on the CIPP model, this study was conducted in 2020. In this study, context, input, process, and product conditions were evaluated from the perspective of the department manager, faculty members, and assistants on the order of the educational deputy. The present study aimed to determine the desirability of the presented programs and inform the managers regarding the current situation of the Pediatrics educational department to help decision-makers revise the program for providing the needs better in the future.

Materials and Methods 
The current study is an evaluation descriptive-analytical study conducted in 2020 in the Pediatrics Department of Hormozgan University of Medical Sciences. According to the CIPP model, program evaluation was performed in four areas: context, input, process, and product.

Participants
The study population consisted of 40 faculty members, first, second, third-year assistants, and the Pediatrics department director. The census selected the samples.

Instrument
Data collection was done by the modified researcher-made questionnaire of Limuei et al in the internal evaluation of Tabriz dental school using the CIPP model in 2015 (16). Experts’ opinions were utilized to confirm the face and content validity of the questionnaire by providing research tools to 10 faculty members of the Department of Pediatrics and medical education specialists. They were asked to submit their comments and suggestions for correction. Accordingly, the necessary changes were made. Also, to determine the reliability of the questionnaires, Cronbach’s alpha coefficient was used. The Cronbach’s alpha obtained from the faculty members’ questionnaire was 0.96, and the residents’ questionnaire was 0.93.
The Likert scale was used to determine the satisfaction or desirability of the questions. A score of 5 was given to the best condition and 1 to the most unsuitable one. Other undesirable, relatively desirable, and desirable situations were scored 2, 3, and 4 points. The numerical value of each item was determined using the mean value and standard deviation tests. Then, using the judgmental model of previous studies, the degree of desirability (desirable, relatively desirable, undesirable) of the items and criteria were determined. A score of 1-2.33 was considered undesirable, 2.34- 3.66 relatively desirable, and 3.67-5 desirable. In evaluating the context, the department’s mission, goals, management, and organizational structure were evaluated. Department training programs, educational facilities, and facilities in input evaluation, learning-teaching process, and educational research activities evaluated the process and abilities of graduates and faculty members in evaluating the product. The results were summarized in tables.

Data analysis
Data analysis was performed in two parts: descriptive and analytical statistics. In the descriptive statistics section, the parameters of mean and standard deviation were used. In the inferential and analytical sections, after confirming the normality of the data, the data were analyzed using the SPSS Kolmogorov-Smirnov test using SPSS software version 20. The criteria for students’ inclusion in the study were willing to participate in research, passing the clinical training course, or completing the course. The exclusion criteria were dissatisfaction with participation in each stage of research. During the data collection, the participants’ work process was not disrupted. They were given the written consent to participate in the research; they were assured that all their information would be kept confidential at each stage of the research. In the present study, since access to graduates was not possible, faculty members and assistants were utilized to complete a questionnaire. The ethics committee approved this project of Shiraz University of Medical Sciences with the code 1399.750.

Ethical considerations
Participants in this study signed the written informed consent and were reminded that they could withdraw from the study at any time, in addition to adhering to the principle of confidentiality. The University Ethics Committee has approved this study with the code IR.SUMS.REC.1399.750. In terms of specific ethical principles on program evaluation, the integrity, accountability, respect, and beneficence of this program were comforrmed by educational and ethical experts in Shiraz ansd Hormozgan university of meical sciences. 

Results
Regarding the participants, the descriptive findings revealed that, out of 20 faculty members, 8 (40%) were female, and 12 (60%) were male. Of the 20 assistants, 10 (50%) were female, and 10 (50%) were male. 70% of the department’s faculty members (14 people) had the rank of assistant professor, and 30% (6 people) had the rank of associate professor. Among the faculty members, 42% had less than ten years of teaching experience, and 58% had over ten years of experience. Considering the information associated with the quality of job rank and teaching experience of faculty members regarding the internal evaluation standard of the Quality Assessment Center of University of Tehran, the degree of desirability in both cases was relatively good.
Analytical findings showed that according to the amount (t=3.63) and the level of significance (p= 0.001), the overall evaluation of the respondents of the Pediatrics residency program of Hormozgan University of Medical Sciences based on the CIPP model among faculty members and residents has a statistically significant difference. Reference to descriptive statistics showed that the average score of professors ( = 3.15) and the score of assistants 
( = 2.57) could be stated; faculty members have a more positive attitude towards students in evaluating the child support program. This test also examined the evaluation of faculty members and assistants of the pediatric assistant program based on the CIPP model in each of the areas of context, input, process, and product, the results of which are listed in table 1. According to the results, faculty members in all curriculum areas except the product area had a significantly better evaluation than the residents. In terms of product, the evaluation of faculty members and assistants of the training program was not significantly different (p=0.398).

Table 1. Comparison of the average evaluation of the residency training program from the perspective of faculty members and residents

Area

Department

Mean

SD

df

t (Sample t-test)

p-value

Context

Residents

2.50

0.57

38

3.79

0.001

Faculty members

3.17

0.53

Input

Residents

2.52

0.57

38

2.56

0.015

Faculty members

3.01

0.63

Process

Residents

2.72

0.57

38

3.71

0.001

Faculty members

3.35

0.50

Product

Residents

2.60

0.63

38

0.855

0.398

Faculty members

2.76

0.51

Total Evaluative

Residents

2.57

0.50

38

3.63

0.001

Faculty members

3.15

0.50

The evaluation of context 
The factors studied in the context dimension include the department’s mission and goals and management, organizational structure, and department organization. Findings revealed that the level of knowledge of faculty members and assistants regarding the department’s goals and mission, in general, is relatively desirable. The appropriateness of goals with individual needs, students’ expectations, and community needs in the department’s mission were relatively desirable. In reviewing the goals and mission of departments, there was not much satisfaction from the students and their estimation, in this case, was undtesirable. Findings related to management and organizational structure, in general, indicated a relatively good evaluation. However, cases such as the extent to which the department manager used residents’ views in implementing programs from the perspective of residents and department costs from the perspective of faculty members had the lowest average and undesirable scores (Table 2).

Table 2. Context evaluation results

Dimension

Department

Number of questions

Number of respondents

Mean value

SD

Judgment

Mission and goals of the department

Faculty members

13

20

3.23

0.59

Relatively desirable

Residents

12

20

2.49

0.59

Management and organizational structure

Faculty members

33

20

3.15

0.56

Relatively desirable

Residents

28

20

2.51

0.59

General judgment about the context dimension

Faculty members

46

20

3.17

0.53

Relatively desirable

Residents

40

20

2.50

0.57

The evaluation of input 
The criteria studied in the input dimension included educational programs, learning, educational resources, department equipment and facilities, faculty members, and students. The training programs for each semester were implicit in the department, and the training programs in the department were relatively good for creating a scientific base, improving skills, commitment, and medical ethics. Still, there was not enough proportion between theoretical and practical courses for the residents. Faculty members and assistants had little involvement in developing the programs. Access to computer and the internet facilities was approximately desirable and somewhat undesirable. The suitability of domestic and foreign scientific books and journals in the library with the scientific needs of the department was relatively desirable. The facilities available in the department in terms of library space and classrooms equipped with video projectors and teaching aids were evaluated as relatively desirable and somewhat undesirable. Generally, the index of educational facilities and facilities from the perspective of assistants and faculty members was reported as undesirable and relatively desirable (Table 3).

Table 3. Input evaluation results

Dimension

Department

Number of questions

Number of respondents

Mean value

SD

Judgment

Department educational programs

Faculty members

13

20

3.23

0.6

Relatively desirable

Residents

20

20

2.58

0.620

Educational facilities

Faculty members

10

20

2.71

0.8

Relatively desirable

Residents

8

20

2.37

0.693

Undesirable

General judgment about the input dimension

Faculty members

23

20

3.01

0.6

Relatively desirable

Residents

28

20

2.52

0.569

The evaluation of process 
Factors studied in the process dimension included the teaching and learning process and educational research activities. Findings showed that the lessons taught are part of the approved curriculum of the department. In teaching, students’ prior information is relatively considered, and their differences in teaching are taken into account. The interest of faculty members in teaching was desirable, and providing feedback in the department was considered undesirable by the assistants. The involvement of faculty members in research activities was relatively desirable, and the extent of their cooperation in guiding and advising dissertations was reported to be desirable. Also, the level of cooperation in the department and out of department research activities was relatively desirable, but funding for research projects was undesirable (Table 4).

Table 4. The process evaluation results

Dimension

Department

Number of questions

Number of respondents

Mean value

SD

Judgment

Teaching-learning process

Faculty members

17

20

3.43

0.51

Relatively desirable

Residents

17

13

70.2

0.53

Research-educational activities

Faculty members

9

20

19.3

0.53

Relatively desirable

Residents

9

13

74.2

0.74

General judgment about the process dimension

Faculty members

26

20

35.3

0.5

Relatively desirable

Residents

26

13

72.2

0.56

The evaluation of product 
Factors studied in the product dimension included the academic products of graduates and faculty members. The rate of authorship or translation of the book was reported to be undesirable. The rate of publication of scientific articles in prestigious domestic and foreign journals was relatively desirable. The number of papers presented by faculty members in domestic and foreign conferences was considered undesirable (Table 5).

Table 5. The product evaluation results

Factor/Criterion

Department

Mean

SD

Judgment

Graduates

Faculty members

2.76

0.51

Relatively desirable

Residents

2.60

0.63

Relatively desirable

Scientific productions of faculty members

 

Compiled or translated a book in the last three years

Undesirable

Number of articles published in prestigious journals in the last three years

Relatively desirable

Number of papers presented in domestic and foreign seminars in the last three years

Undesirable

           

 

Discussion
This study was aimed to evaluate the effectiveness of the pediatrics residency training program at Hormozgan University of Medical Sciences through the perspectives of Pediatrics residents and 20 faculty members. The selection of the context, input, process, product in the CIPP appraisal model- out of the various evaluation models- was based on the reflection that with respect to the CIPP model offers direction in the form of checklists and flow diagrams that can be used in evaluating a program as a whole through four major components. The results of four areas of context, input, process, and product of the educational program of the department of Pediatrics of Hormozgan University of Medical Sciences in 2020 were evaluated as relatively desirable from assistants’ perspectives and faculty members. This other study’s context, input, and process status are consistent with Zandovaniyan’s study (21). From the perspective of faculty members and assistants, the context dimension was relatively desirable. The department management structure and financing had the highest and lowest mean scores among the context domain indicators. 
In other words, the strength of the management criterion and organizational structure from the participants’ point of view is the department’s management structure, and its weakness is financing. It seems that to solve this problem, senior university administrators should have a unique look at financing the Pediatrics department. In line with the results of this study, Limuei et al also stated that more planning is needed to finance the departments (16). Moreover, these results were in line with other studies in Tehran University of Medical Sciences, in which the results showed the undesirable condition of context, process, and product area and undesirable state for input except for “interest and understanding of students towards field and labor market” factor, which had a fairly desirable situation (22). In the field of ​​input, four factors of faculty members and learners, educational programs, and facilities were examined from the perspective of faculty members and assistants.
The factor of educational programs was considered relatively desirable from the perspective of faculty members and assistants. In this regard, Jamali et al, in the internal evaluation of Shahrekord Midwifery Department (18) stated: acting to increase student participation in the educational and evaluation programs, encouraging the professors to interact more with students to increase educational and research activities, are of the proposed solutions for this field. From faculty members’ point of view, the educational facilities of the department were considered relatively desirable and undesirable from the residents’ point of view. In other words, the students were not satisfied with the existing educational facilities and wanted to improve them. In terms of input and indicators in its context, the highest and lowest mean scores belonged to the criteria of educational programs and educational facilities, respectively. The results of the internal evaluation of Bangladeshi universities also showed that limited resources and insufficient facilities are the main challenges of quality improvement in higher education centers of Bangladeshi universities (23).
In the process dimension, two factors of the teaching-learning process and educational research activities in the department were evaluated. From the faculty members and assistants’ point of view, the factor of the teaching-learning process was relatively desirable. It was found that students’ differences in teaching are taken into account, and the interest of faculty members in teaching is desirable. Providing feedback to the department was considered undesirable by the assistants. The students’ activities in the context of research activities and participation in solving department problems were weak, which requires more planning and attention of managers and decision-makers. The faculty members and assistants of the Pediatrics department assessed the situation of the process dimension as relatively desirable. The results of Alimohammadi’s studies in the evaluation of Rafsanjan University of Medical Sciences with the model of SIP (25), ethics and colleagues in evaluating the quality of educational programs in higher education and Salimi et al’s study are consistent in evaluating the programs of the orthodontics and restorative department of Hormozgan University (24).
In the context of the product, two factors of scientific production of the department and graduates were evaluated. Evaluation of the department’s scientific production factor, articles published in prestigious domestic and foreign journals, authorship or translation of books, participation in national and international conferences were among the criteria examined in the scientific production factor. Examining the factors, criteria, and indicators, the product dimension was relatively desirable from the perspective of faculty members and assistants. From the perspective of faculty members and residents, the research activities were relatively desirable, the level of cooperation of faculty members in supervising and advising the dissertations was desirable, and the level of cooperation in department and extracurricular research activities was relatively desirable.Still, funding for research projects was deemed undesirable, which requires special attention to research activities associated with this crucial area. Nagata et al’s study showed that the nursing curriculum was not appropriate in input, the number of professors, facilities, and electronic equipment such as the computer systems (25).
Since comprehensive and apparent information on the number of articles or participation in foreign and domestic conferences was not available to the researcher, most of the questions of this criterion were not answered by assistants via researching the information networks and asking the research assistants of the group. Stating the standards provided by the Quality Assessment Center of the University of Tehran, the scientific productions of the faculty members were generally considered relatively desirable. Afshari states that one of the research problems is that there are no separate teaching and research faculty members in the faculties, and the research activities of faculty members are overshadowed. And the high educational burden from their and graduate students’ perspectives is one of the critical obstacles to conducting research (26). According to Bernardin, most faculty members cite lack of time as a problem in conducting research, as most of their time is spent on training and providing clinical services. At the same time, he believes that a faculty member can sometimes do their job successfully, be a good researcher, and use the results of their research in teaching (27).
As previously mentioned, due to the lack of access to graduates, faculty members and assistants were used to complete questionnaires related to graduates. Faculty members and assistants rated the graduates as relatively desirable. The results revealed that the context of graduates is generally in a relatively desirable condition but needs more attention. As mentioned earlier, the product was relatively desirable from the faculty members’ and assistants’ perspectives. These results are consistent with the results of a studies by Jannati et al in Tabriz 2017 and Rooholamini et al in shiraz 2017 (28,29).

Conclusion
This study showed that the overall pedagogical education program of Hormozgan University of Medical Sciences is in a relatively good condition. Therefore, considering that the authorities want to train experienced and efficient physicians in the medical community, it is worth paying more attention to providing educational facilities, paying great attention to research activities, eliminating shortcomings and deficiencies. The results of this study can guide educational administrators and vice-chancellors and university planners to improve the quality of the curriculum, which will have a direct impact on improving the ability of child support students. Due to the limitations of this study in non-intervention, it is suggested that in the subsequent studies of the Pediatrics department, intervention should be done to address the department’s shortcomings. After presenting the results of this study to the relevant units, first, the areas without the required standard have reached the standard. The areas with the standard also help eliminate the shortcomings and plan to maintain optimal conditions and prepare standards at higher levels. Finally, the pediatric asisstants and facilities indicated that, the organization is not stagnant and has a dynamic movement. To ensure the quality of education, programs should be reviewed continiously and evaluated periodically since evaluation as one of the tools of quality improvement makes it possible to identify the strengths and weaknesses of the programs. Appropriate and effective steps should be taken to create change and improve the process by strengthening the positive aspects and eliminating the shortcomings. Researching only one university and the impossibility of intervention is a limitation for the generalization and application of the findings. Also lack of access to graduates due to the time and place of the study were other limitations of this study.

Acknowledgements
The authors consider it necessary to appreciate the cooperation of experts and participants in the study. This project has been done with the financial support of Shiraz University of Medical Sciences, Shiraz, Iran, with the number 20841.

  1. Toosi M, Modarres M, Amini M, Geranmayeh M. Context, input, process, and product evaluation model in medical education: a systematic review. J Educ Health Promot 2021 May 31;10(1):199. https://pubmed.ncbi.nlm.nih.gov/34250133/
  2. Cook DA, Kuper A, Hatala R, Ginsburg S. When assessment data are words: validity evidence for qualitative educational assessments. Acad Med 2016 Oct 1;91(10):1359-69. https://pubmed.ncbi.nlm.nih.gov/27049538/
  3. Aziz S, Mahmood M, Rehman Z. Implementation of CIPP model for quality evaluation at school level: a case study. J Educ Educational Dev 2018 Jun;5(1):189-206.
  4. Hemati Z, Irajpour A, Allahbakhshian M, Varzeshnejad M, Abdi SM. Evaluating the neonatal intensive care nursing MSc program based on CIPP model in Isfahan university of medical sciences. Iran J Med Educ 2018;18(0):324-32.
  5. Nasr VG, Guzzetta NA, Mossad EB. Fellowship training in pediatric cardiac anesthesia: history, maturation, and current status. J Cardiothorac Vasc Anesth 2019 Jul 1;33(7):1828-34. https://pubmed.ncbi.nlm.nih.gov/30243872/
  6. Amuei F, Mahmodi E. Study of the education and research quality in basic sciences through internal evaluation in two consecutive educational years in MUMS. Res Med Educ 2019 Mar 10;11(1):20-8.
  7. Da Dalt L, Anselmi P, Furlan S, Carraro S, Baraldi E, Robusto E, et al. An evaluation system for postgraduate pediatrics residency programs: report of a 3-year experience. Eur J Pediatr 2017 Sep;176(9):1279-83. https://pubmed.ncbi.nlm.nih.gov/28762071/
  8. Hakan K, Seval F. CIPP evaluation model scale: development, reliability, and validity. Proced-Soc Behav Sci 2011 Jan 1;15:592-9.
  9. Makarem A, Movahed T, Sarabadani J, Shakeri MT, Asadian Lalimi T, Eslami N. Evaluation of educational status of oral health and community dentistry department at Mashhad dental school using CIPP evaluation model in 2013. J Mashhad Dent School 2014;38(4):347-62.
  10. Young Lee S, Shin JS, Lee SH. How to execute context, input, process, and product evaluation model in medical health education. J Educ Eval Health Prof 2019;16:40. https://pubmed.ncbi.nlm.nih.gov/32299190/
  11. Seif A. Educational measurement, evaluation, and evaluation. Tehran: Doran Publications. 2004;128.
  12. Kool B, Wise MR, Peiris-John R, Sadler L, Mahony F, Wells S. Is the delivery of a quality improvement education programme in obstetrics and gynecology for final year medical students feasible and still effective in a shortened time frame? BMC Med Educ 2017 May 26;17(1):91. https://pubmed.ncbi.nlm.nih.gov/28549464/
  13. Bilan N, Negahdari R, Hazrati H, Moghaddam SF. Examining the quality of the competency-based evaluation program for dentistry based on the CIPP model: a mixed-method study. J Dent Res Dent Clin Dent Prospects 2021 Summer;15(3):203-9. https://pubmed.ncbi.nlm.nih.gov/34712412/
  14. Lippe M, Carter P. Using the CIPP model to assess nursing education program quality and merit. Teaching Learning Nurs 2018 Jan 1;13(1):9-13.
  15. Tabari M, Nourali Z, Khafri S, Gharekhani S, Jahanian I. Evaluation of educational programs of pediatrics, orthodontics and restorative departments of babol dental school from the perspective of the students based on the CIPP model. Caspian J Dent Res 2016;5(2):8-16.
  16. Limuei, F. Internal evaluation of educational departments of Tabriz school of dentistry using the CIPP model in 2015. Irden. 2015
  17. Mohamadi N, Hasanian M, Borzou S, Amini R, Sajedi ZH, Omidi A, et al. The comparison of internal evaluation of 4 nursing departments in nursing and midwifery faculty of Hamedan university of medical sciences. Avicenna J Nurs Midwif Care 2013;21(1):66-76.
  18. Jamali S. Internal evaluation of midwifery department, nursing and midwifery faculty of Shahrekord university of medical sciences. Educ Strat Med Sci 2019 Jan 10;11(5):191-9.
  19. Sarkar SH, Hossain SZ. Budgetary challenges of higher education: evidence from Bangladesh. J Educ Pract 2016;7(12):32-41.
  20. Keshmiri F, Heydari AS. Evaluating the students evaluation in residency program based on competency-based approach. J Med Educ Develop 2019 Aug 10;14(2):74-82.
  21. Zandovaniyan A. The evaluation of teacher training center in Khuzestan province using the CIPP evaluation model. J Educ Psycol Shahid Chamran Ahwaz Univ 2006;13(2):137-62
  22. Neyazi N, Arab M, Farzianpour F, Majdabadi MM. Evaluation of selected faculties at Tehran university of medical sciences using CIPP model in students and graduates point of view. Eval Program Plann 2016 Dec 1;59:88-93. https://pubmed.ncbi.nlm.nih.gov/27639657/
  23. Kader M. Curriculum evaluation of professional post-graduate MA in ELT/TESOL program: a study on Bangladeshi institutions (Doctoral dissertation, BRAC University).
  24. Alimohammadi T, Rezaeian M, Bakhshi H, VaziriNejad R. The evaluation of the medical school faculty of Rafsanjan university of medical sciences based on the CIPP model in 2010. J Rafsanjan Univ Med Sci 2013 May 10;12(3):205-18.
  25. Nagata S, Gregg MF, Miki Y, Arimoto A, Murashima S, Kim MJ. Evaluation of doctoral nursing education in Japan by students, graduates, and faculty: a comparative study based on a cross-sectional questionnaire survey. Nurse Educ Today 2012 May;32(4):361-7. https://pubmed.ncbi.nlm.nih.gov/21665333/
  26. Afshari P, Haidarpour S. Is “teaching” an obstacle to “research”? Iran J Med Educ 2002 Dec 10;2:9-.
  27. Bernardin HJ. Academic research under siege: toward better operational definitions of scholarship to increase effectiveness, efficiencies, and productivity. Hum Resource Manag Rev 1996 Sep 1;6(3):207-29.
  28. Jannati A, Gholami M, Narimani M, Kabiri N. Evaluating educational program of bachelor of sciences in health services management using CIPP model in Tabriz. Depiction Health 2017 Sep 21;8(2):104-10.
  29. Rooholamini A, Amini M, Bazrafkan L, Dehghani MR, Esmaeilzadeh Z, Nabeiei P et al. Program evaluation of an integrated basic science medical curriculum in Shiraz medical school, using CIPP evaluation model. J Adv Med Educ Prof 2017 Jul;5(3):148-54. https://pubmed.ncbi.nlm.nih.gov/28761888/