Abstract 
Background: The present study aimed at investigating the effect of a stigma reduction intervention package on improving the attitude of medical students towards patients with patients with psychiatric disorders.
Methods: The authors included medical students at the Iran Psychiatric Hospital in the study and divided them into two intervention and control groups using cluster randomization method. In addition to regular psychiatric training, the intervention group was trained with a package that included watching a film, stigma awareness workshop, and direct social contact with patients with psychiatric disorders; each followed by group discussions. The outcome measures were assessed three times; before, immediately after, and three months after the intervention, using the Social Distance Scale (SDS), Dangerousness Scale (DS), and the short form of Opening Minds Scale for Health Care (OMS-HC). 
Results: 74 filled all the questionnaires in all three times; the intervention group included 39 and the control group included 35 students. In one-month follow-up, stigma among the participants decreased according to OMS-HC and DS questionnaires as the difference between stigma scores were significant (p-value<0.001); however, based on SDS questionnaire, the decrease in stigma was not significant (p-value=0.074). In the three-month follow-up, stigma score decreased only based on the differences of OMS-HC scores which were significant between two groups (p-value=0.020).
Conclusion: Attending psychiatry clinical rotation could improve the students’ attitudes towards patients with psychiatric disorders and based on OMS-HC questionnaire if combined with intervention package, could improve this attitude even further.
Keywords: Education, Mental disorders, Medical students, Medical education, Social stigma

Introduction
Stigma originates from Greek terminology, referring to physical labeling or marking for exposing immoral or socially unacceptable people. Throughout the years, the meaning of stigma changed from merely a physical sign to becoming a social process, characterized by disapproval and separation. Although stigma is a worldwide problem, its experiences vary by country, culture or profession. For instance, it is shown that stigma in Asian countries tend to be higher than Western countries (1,2).
Stigma is imposed on people or groups with certain characteristics; race, gender, income level and health conditions. Among these groups, patients with psychiatric disorders have been affected by stigma and its negative effects for ages. According to the World Health Organization’s data, around 1-2% of the world’s population suffers from severe and 10-15% from moderate mental illness. However, nearly two-thirds of them do not seek treatment. One main reason for this under-treatment is stigma and the way society treats patients with psychiatric disorders (3). In Iran, it has been estimated that the prevalence of psychiatric disorders varies from 17.1 to 23.6% in different parts of the country (4). 
Stigma towards psychiatric patients leads to severe health problems, adds extra pressure on them and reduces their adherence to the treatment; moreover, it reduces the mental well-being of the society and causes a delay in treatment seeking among the general population. Therefore, the stigma adds an excessive burden to both the patients and society (5). 
Additionally, stigma towards psychiatric patients is present among different professions, including healthcare providers (6). It is reported that nine out of ten patients with psychiatric disorders have been stigmatized, even by medical staff (7). This stigma from the medical staff, which is not limited to psychiatric disorders, derives from various factors including; negative attitudes and believes towards patients with certain conditions, fear of infection or harm, lack of knowledge about the medical condition and the way they are clinically managed (6).
Negative attitudes of healthcare providers adversely affect patients and add more pressure on them. As a result, patients’ willingness to begin or continue the treatment process decreases and earlier quits of treatment happen. Stigma also puts healthcare providers at a greater risk to neglect their own mental wellbeing and treatment seeking (6,7). 
Considering the importance of tackling mental health related stigma, it is crucial to design and implement effective interventions to improve the knowledge, attitude and practice of medical students towards psychiatric patients. In this regard, many interventions have been developed and many media have the capacity to address this issue (8). Despite the divergent data on the level of effectiveness for interventions, there are studies that indicate increased contact with psychiatric patients and improved knowledge about disorders could improve the attitude of healthcare providers (9). In order to maximize the effectiveness of these contacts, five groups of ingredients are suggested; proper design, specified target group, presenters with lived experiences, diverse messages, and discussing follow-up with participants (10).
In Iran, the stigma towards psychiatric patients exists; which is also evident among Iranian healthcare workers (6). In order to address this issue, some studies were carried out among different groups of health care providers; Vaghee et al (11) used contact-based and acceptance and commitment training for reducing stigma among nursing students and found that both ways are effective; Asayesh et al (9) utilized an anti-stigma program among nursing students which proved to be effective in stigma reduction; also, Amini et al (12) implemented an intervention which increased the time that medical students have increased contact with psychiatric patients and found that it had no significant impact on reducing stigma. Since these studies showed different findings and in order to reduce the medical students stigma effectively, an interventional package, designed by Rezvanifar et al (8), was used in this study; which is consisted of educational content on stigma and its effects on health outcomes, in addition to direct social contact with patients who have mental health disorders.
Therefore, this study was designed to find the effectiveness of this intervention package on reducing the stigma of medical students towards psychiatric patients, and compared it to the routine psychiatric rotations in medical curricula.

Materials and Methods
Overview and participants
A Cluster-randomized design was used to assess how anti-stigma interventions could change the stigma of medical students towards psychiatric patients. To do so, changes in stigma scores was analyzed between the students who passed the routine psychiatric rotation in xxx and the students who went through the intervention package in addition to the routine course programs.
The study population consisted of medical students who attended their clinical rotation of psychiatry from November 2021 to June 2022 in “Iran Psychiatric Hospital”; each rotation continued for four weeks. Sample size was estimated to be 80. All the students with the abovementioned criteria were eligible to participate in the study with no limitation for age and number of passed semesters; however, students who were not willing to participate in the study or had taken psychiatry clinical rotations in the past were excluded. Due to the number of students who entered the clinical rotations during the mentioned period, 93 participants were included in the study.
In each four-week rotation, around 20 to 30 students were assigned by the university administration to “xxx Hospital”; to cover the calculated sample size, four groups of students entered the study. To divide the participants, one-stage cluster sampling method was used where each group of students was considered as a cluster; then using a computer-based randomization model, the groups were divided into two intervention and two control groups.
Informed consent was obtained from the participants upon their entry and questionnaires were filled anonymously and the participants could quit the study upon their desire. This study was approved by the Research Ethics Committee the Iran University of Medical Sciences and was conducted based on the Helsinki Declaration. 
 
Measures
Electronic online forms were designed to gather demographic data and answers to the questionnaires. Demographic factors included sex, age, number of clinical semesters passed, marriage (single, married, divorced or widowed), place of residence at the time of intervention (alone, with first degree family, in dormitory or with other family members), history of having psychiatric treatment (whether it was in the form of using medications or having psychotherapy sessions) in the person and any of the family members and history of personal psychiatric hospitalization.
Three questionnaires were used to measure stigma among the participants:

The Opening Minds Scale for Healthcare Providers (OMS-HC): This scale was designed by Kassam et al (13), in 2012, to answer the need for a specific measure that works for assessing stigma, particularly among healthcare workers; it is a proper tool for the assessment of stigma reduction programs. The original questionnaire is consisted of 20 questions which are measured with a 5-point Likert scale (1=I completely disagree, 2=I disagree, 3=neither agree nor disagree, 4=I agree, and 5=I completely agree). However, a short form of the questionnaire containing 10 initial questions was used in this study; ranging from 10 to 50. Kordlou et al reported its test-retest reliability coefficient 0.7 and Cronbach’s-alpha 0.76 in their study (2,14).

Dangerousness Scale (DS): This questionnaire measures the respondents’ attitude about the dangerousness of patients with psychiatric disorders (15). This tool contains eight questions and is scored using a seven-point Likert scale from one to seven, with the lowest score being seven and the highest score being 56. For this questionnaire, Cronbach’s alpha coefficient was 0.96, test-retest reliability coefficient was 0.88 and content validity coefficients was 0.77 (2,16).

Social Distance Scale (SDS): The SDS measures the respondent’s willingness to interact with a patient suffering from psychiatric disorders. SDS was introduced by Bogardus who proposed a measure to assess the attitudes toward particular races and ethnicities; the main concept is that the more negative the attitude towards a group is, the more that person wishes to keep a distance from that group. It was first used by Cumming and Cumming in 1957 for the assessment of attitudes toward psychiatric patients; and then various changes such as adding a vignette and Likert scale were made in the following years (17). This questionnaire contains 7 questions and is scored using a four-point Likert scale from zero to three for each question and a total of zero to 21. Cronbach’s alpha coefficient was 0.92, test-retest reliability coefficient was 0.89 and content validity coefficient was 0.75 (16). In all the questionnaires, higher score indicates higher stigma. English versions of the questionnaires are available in Supplement 1.
The choice of demographic factors was based on the authors’ previous experience in the field of psychiatry and stigma research; regarding the questionnaires, OMS-HC is designed specifically to assess the programs aimed at reducing psychiatric disorders stigma among healthcare workers (13); perceived dangerousness affects the emotional components of stigma and in addition to SDS which measures the tendency of people to keep their distance from the psychiatric patients, are indicators of stigma (18); in case of medical students, the higher perceived risk of dangerousness, the higher support for segregating the patients and discriminatory behavior.

Sample size
As OMS-HC was our main questionnaire which directly assesses the stigma among healthcare providers, the related study and metrics were used to calculate the sample size. Based on the findings of a culturally relevant study (14), the minimum sample size for each group was equal to 38. Since a cluster randomization method was used and the participants were allocated by administrative authorities, the numbers in control and intervention groups were not equal; however, each group has more than 38 participants.

Procedure, design and intervention package
Prior hypothesis was considered as having a significant difference between the stigma scores of control and intervention groups throughout the study, and in the three months follow-up, for each questionnaire separately. Another hypothesis was specified as all the demographic factors and being in the intervention group would affect the change in stigma score.
After being assigned to one of the groups, all the participants filled out the study questionnaires through online forms at the start (T1), right after the end (T2) and 3 months after the end of their psychiatric rotation (T3). Participants of the control group only attended the four-week routine activities of psychiatric rotation, which included clinical rounds, educational classes and morning reports on the topics regarding interactions between doctors and patients, diagnosis and treatment of psychiatric disorders. During these weeks, students were present in outpatient clinics, and inpatient visits, putting them through indirect contact with patients.
Participants of the intervention group went through the same routine. In addition, they were trained with an intervention package, which was designed by assessing the most effective anti-stigma interventions, published in the literature and gathering expert opinions. According to the criteria for quality contact-based programs, the intervention covered all five main domains (design, target, staff, message and evaluation/follow-up) suggested by Corrigan et al (10). In a previous study, by Rezvanifar et al (8), to find the most effective interventions to improve the attitudes of medical students, the related literature was reviewed and the extracted data was presented to the expert panel as they selected the best interventions for a uniform package, based on Delphi method. Recommended activities consisted of showing a stigma related movie, psychiatric training on stigma, social communication and direct contact with people who are affected by psychiatric disorders, and finally, implementing group discussions. In this study, intervention was designed accordingly and the recommended activities were implemented for three consecutive weeks: 1- Students watched an Iranian movie “Baradaram Khosro” (“My brother Khosro”) focusing on the life of a patient with bipolar disorder and his interactions with his family and society, and discussing the movie afterwards; main focus of this session was for them to understand the effect of stigma on lives of people with psychiatric disorders; this session lasted for about three hours. 2- Next week, they attended a two-hour workshop, trained by a psychiatrist, where the meaning of stigma, its effects and the ways to tackle it were discussed in more details and real-life experiences were shared anonymously. 3- On the third and final week, students had direct interactions with patients who were discharged from the hospital and were on their rehabilitation period at the time of intervention; the participants collaborated with them on making handiwork and playing sports like volleyball; at the end, a discussion session was held to summarize the experiences. The procedure is illustrated in Figure 1 and full details are available in supplement 2.
In order to reduce contamination about the intervention and the information which was given to the intervention group, some measures were taken; members of a group had no direct contact with other groups in the hospital setting; since students spent all of their course time for about one month in the Iran Psychiatric Hospital which only offers psychiatry clinical rotations, and the information sharing was limited. In addition, all the participants were encouraged not to disclose the information regarding the structure and content of the workshops to others.

Ethics approval and consent to participate
Written informed consent was obtained from the participants upon their entry and questionnaires were filled anonymously and the participants could quit the study upon their desire. This study was approved by The Research Ethics Committee of the Iran University of Medical Sciences (Ethical code: IR.IUMS.FMD.REC.1400.059) and was conducted based on the Helsinki Declaration. 

Statistical analysis
Data was tabulated and analyzed using Stata software version 14. To assess the normality, histograms and Shapiro-Wilks test was used. Frequencies and percentages were used to describe categorical data; mean and standard deviation (SD) was used for continuous data. To examine the differences between the subgroups, independent samples T-test and Chi-square were used where appropriate. ANCOVA and repeated measures ANOVA were used to investigate the effect of intervention. Leven’s test was utilized to test the homogeneity of variances assumption. Level of confidence was estimated at 95%.
Multiple linear regression was used to investigate the associations between the variables and variables with p-value larger than 0.2 in univariate analysis were entered into the multiple regression model with backward-Wald method.

Results
Demographic characteristics
Ninety-three students entered the study and were randomly assigned to 2 groups; 51 participants (54.8%) were in intervention group and 42 (54.2%) were in the control group. Thirty-nine students were male (41.9%) and 54 were female (58.1%). Mean age of the participants was 23.5±1.48; eighty-six were single (92.5%) and 37 (39.8%) were living with their first-degree family. Demographic characteristics are mentioned in details in table 1.
In the initial phase (T1), 93 participants filled out all three questionnaires. In the second phase (T2), 80 students (86%) filled out the questionnaires; among them, 36 (45%) were in the control group and 44 (55%) were in the intervention group. At the third phase (T3), three-month follow-up, 74 (79.5%) completed the study; from which 35 (47.3%) were in the control group and 39 (52.7%) participated in the intervention. Table 2 shows the mean and SD score for each questionnaire in each time.
There were 19 participants who did not participate in the third phase of the study (7 in the control group and 12 in the intervention group). The comparison of demographics and questionnaire scores in the first phase is presented in table 3.

Table 1. Demographic characteristics of the participants.

* Chi-square or T-test.

Table 2. Distribution of scores in the study population at three time points

¹: Opening Minds Scale for Health Care Providers. ²: Dangerousness Scale. ³: Social Distance Scale. * T₁: start of the clinical rotation, T₂: right after the end of rotation, T₃: three months after the end of psychiatric rotation. ** Results of the comparisons between control and intervention groups in each time using independent samples t-test.

Table 3. Comparison of the participants remaining until the end of the study and those who have dropped off

Comparison of intervention and control group 
The scores for OMS-HC, DS, and SDS were analyzed to determine whether there was a significant difference in students’ attitudes towards psychiatric patients based on the intervention and control group scores over time. For all three questionnaires, the time effect was significant (p-value<0.001); showing that stigma decreased during the study time regardless of the groups. For both OMS-HC and DS questionnaires, the effect of group was also significant (p-value= 0.032 and 0.002 respectively). This indicates that the improved attitude in the intervention group was significantly higher than the control group, but not for the SDS (p-value=0.266). The effect of time and group interaction was significant for OMS-HC and DS (p-value<0.001), but for the SDS, it was not significant (p-value=0.260).
Between group analysis in each time was analyzed separately as well; according to the results, students of both groups had similar levels of stigma as there were no significant difference between the control and intervention group in the beginning of the intervention (T1) (p-value>0.05); in the second time (T2), according to OMS-HC, DS questionnaires, students in intervention group had lower stigma as the difference between stigma scores were significant (p-value<0.001), however based on SDS questionnaire, this difference was not significant (p-value=0.074). Finally in the third assessment (T3), participants in the intervention group had lower stigma based on OMS-HC scores, which demonstrated a significant difference between two groups (p-value=0.020) and other two questionnaires indicated no significant difference (p-value=0.789 for DS and 0.778 for SDI). Distribution of the scores at each timeframe is presented in table 2.
ANCOVA was applied to evaluate the change of attitudes based on the scores for questionnaire between groups, considering the baseline covariates. For OMS-HC, the Leven’s test was not significant (p=0.200). OMS-HC at T3 was positively associated with the score at T1 (p=0.005), and the intervention group had lower score after controlling for covariates (p=0.015), showing reduced stigma. No other variable was significantly associated with the score at T3.
For DS questionnaire, the Leven’s test was not significant (p=0.701). DS score at T3 was not significantly correlated with the score at T1 (p=0.703), and the scores of control and intervention groups were not significantly different (p=0.720).
For SDS questionnaire, the Leven’s test was not significant (p=0.994). SDS score at T3 was not significantly correlated with the score at T3 (p: 0.704), and the scores of control and intervention groups were not significantly different (p=0.721).

Factors affecting students’ attitude
Linear regression was used to find effective factors in improving students’ attitudes, three months after the intervention. Outcome variables were defined as the difference between the stigma score three months after the intervention (T3) and the score before the start of intervention (T1) for each questionnaire, making it three outcome variables in total. The independent variables were being in the intervention group, sex, age, number of semesters on clinical rotation, marriage, place of residence (living alone, in dormitory or with family), personal history of psychiatric medication and psychotherapy, familial history of psychiatric medication and psychotherapy were considered as independent ones. As there was only one participant with history of hospitalization, this variable did not enter the model.
For the OMS-HC questionnaire, being in the intervention group [β=2.12, p=0.049, CI=(0.13, 4.23)], for DS, lower age (β=2.34, p=0.005, CI= (0.74, 3.95)] and living alone (β=6.29, p=0.018, CI=(1.12, 11.45)] were the effective factors for reducing the stigma score. Nevertheless, none of the measured factors, significantly affected the difference of scores for SDS questionnaire. 

Discussion
Based on the findings, stigma towards psychiatric patients decreased over time as the time effect was significant for all three questionnaires. Effect of group was significant based on OMS-HC and DS questionnaires but not for SDS. Time and group effect was significant based on OMS-HC and DS questionnaires but again, not significant according to SDS. While comparing the stigma of the participants of groups in each time separately, at the beginning of the intervention, there was no significant difference between control and intervention group and they had the similar stigma level. At the end of the intervention based on OMS-HC and DS questionnaires the difference was significant and at the three-month follow-up, the only significant difference was between OMS-HC scores.
First, the effect of time was observed, which was significant in reducing the stigma score; meaning that regardless of group, students had lower stigma at the end of the study. It could be interpreted that the common program between the groups which is attending the routine psychiatry training could have been effective in reducing the stigma score within this time frame. There are other studies which show similar improvements; a survey of nursing students in Hong Kong showed that although the attitude towards psychiatric disorders was almost positive, it was improved during the training period (19). Another study on nursing students, reached the same conclusion (20). However, there are other studies provide contradicting findings. Studies in Germany and Egypt among medical students and in Iran among nursing students represented that psychiatry clinical training did not affect students’ attitudes (14,21,22). Furthermore, Amini et al (12), assessed a novel educational method containing more hours of interaction between patients and students in psychiatric wards, but did not find a significant improvement in the attitudes of medical students toward patients with psychiatric disorders. Many factors such as the interest of professors and students, amount of interactions with patients and social beliefs, affect students’ prospective towards patients with psychiatric disorders (23,24); since these factors vary from place to place, the clinical rotation was not effective all the time.
Afterwards, the differentiating factor between the groups was the intervention package. Studies show that the interventions with the aim of stigma reduction, can improve the healthcare professionals’ attitudes towards patients with psychiatric disorders in short and medium term (25). Anti-stigma strategies include educational courses to replace misconceptions about mental illness with correct knowledge by giving lectures, discussions, multimedia materials and a variety of direct and indirect interactions with people who have psychiatric disorders (26-28). Studies suggest that these types of mixed methods, similar to the ones that used in the current study, could be the most effective, regardless of their complexity (29). 
However, majority of these interventions are more effective in reducing stigma in short-term (8,30). In a meta-analysis of randomized controlled trials with the aim of reducing different types of stigma among different groups, many of the interventions which were effective in the short-term, lost their effectiveness in the follow-ups after three months or more (31). Rubio et al (32), in their intervention among social worker students reported that after three months, the difference in attitudes and personal stigma between the groups which was present after the intervention was gone in the 3-month follow-up. One of the possible explanations was based on the content of each questionnaire; OMS-HC is a general scale while DS and SDS focus on specific aspects of stigma, and since the intervention does not cover these aspects specifically, the long-term effect is not as clear as the general questionnaire.
Another important aspect of these interventions is the social contact; despite having studies which highlight the importance of these interactions (29), some major concerns exist about this conclusion; for example, weak methodologies, biased reporting, lack of evidence on behavior change, short-lasting effects and lack of measuring dose effect (33). As a result, in spite of having studies showing the positive effect of contact-based interventions (34-36), it is not possible to link the reduction in stigma score, solely to the contact part; thus, it could be deduced that, one possible explanation for the significant decrease in stigma score among the participants in intervention group, compared to the control group, could be the variety of activities, one of them being the quality contact section (10). It is noteworthy that there was no significant difference between the groups in case of social distance score; as the overall SDS score decreased for both groups, it can be concluded that routine psychiatric training is more effective in this regard; also, there should be more focus on addressing social distancing from patients in the interventions.
Another part of the study was assessing the effective factors on reducing the stigma score; lower age and living alone, had significant effect reducing dangerousness score between T1 and T3. Studies revealed that there is no decisive conclusion on the relationship between age and perceived dangerousness (37-39). However, to the knowledge of the authors, none of the studies on stigma reduction interventions among healthcare workers, assessed the effect of age and place of residence on reducing the stigma score.
Furthermore, information exchange between the participants which is called contamination, could converge the scores. In the systematic review conducted by Yamaguchi et al (30), only a small number of studies considered this confounding factor in the design of their research. However, in Kirby’s study (40), this confounding factor’s role was pointed out; and to prevent its effect, the participants were advised not to share the information about their group and the educational content with others. In the current study, the same was done with the participants and asked them not to reveal their group and the interventions which was applied to them. However, this method is not fully secure and the risk of contamination was present; as a result, the reduction in the difference between the two groups in the long-term follow-up, particularly in case of social distancing and perceived dangerousness could be attributed to the shared information between the groups.
The limitations of this study include; being conducted in one center and low number of the participants in each group in addition to not having a third group who have not been in the psychiatric rotation. This study only assessed the knowledge and attitude; however, behavioral changes and discrimination reduction were not assessed. Also, longer period follow-ups could show the long-term effect more robustly. Due to the assignment of students by university, it was not possible to use other methods of randomization. Also, the possibility of contamination could have affected the results as the students have lots of social and academic contacts. The only taken measure was advising the participants not to disclose their group and the interventions which they went through which falls victim to the low certainty; however, some aspects of this interventions, especially direct contact with the patients are effective only if carried out by participants and are somehow immune to contamination. Another limitation is the lack of focus on social distancing from psychiatric patients in the intervention program; as it became apparent, the intervention was not effective in reducing the SDS score; for this matter, a qualitative analysis could have helped to better understand the confounding effect of contamination and better understand the areas where more work is needed including social distancing and dangerousness. Using the shorter version of OMS-HC questionnaire which contained 10 questions instead of 20 could have affected the validity and reliability of this tool. In case of demographic variables, a handful of them was assessed, however, other effective ones such as ethnicity or religion should be studied in future studies. Finally, some participants in both groups were lost due to not willing to participate in follow-ups; their lack of participation can be related to their higher stigma level which effects the final outcome of the study. Thus, in future studies giving better motivation and trying to find the reason for not participating should be carried out.

Conclusion
Attending the psychiatry ward during a routine university program has a positive effect on students’ attitudes; however, anti-stigma intervention package in addition to routine program could reduce the stigma among participants more significantly. Nevertheless, since this difference in the three-month follow-up is only present according to OMS-HC questionnaire, social distance and dangerousness should be addressed more promptly in the package; after addressing all the aspects more comprehensively and implementing further studies that can prove the effectiveness of the intervention on meaningfully reduce the stigma among students, it can be integrated into routine psychiatric education.

Ethics approval and consent to participate
Written informed consent was obtained from participants upon their entry and questionnaires were filled anonymously and participants could quit the study upon their desire. This study was approved by The Research Ethics Committee of the Iran University of Medical Sciences (Ethical code: IR.IUMS.FMD.REC.1400.059) and was conducted based on the Helsinki Declaration. 

Funding
The authors disclose receipt of the following financial support for the research, authorship, and/or publication of this study: This study was funded by Iran University of Medical Sciences (Grant no: 1401-2-23-23753).

Acknowledgement
This study was Dr. Ali Amirkafi’s graduate thesis to Medical Doctor Degree.

Conflict of Interest
There was no conflict of interest in this manuscript.