Effects of the Blended Workshop Learning and Web-Based Learning Sequence on the Learning Level: A New Experiment

Abstract
Background: Background: The blended electronic learning system, raised as a new approach in educational planning with great enthusiasm, seeks novel ways to properly combine the media; however, the sequence of in-person and distance educational methods is an issue neglected so far. This study aims to compare the effect of the sequence of blended workshop learning and web-based learning on the learning level of biostatistics in students who were members of the Student Research Committee at Arak University of Medical Sciences.
Methods: This is a quasi-experimental study with an alternative treatment design. The statistical population consisted of the students who were members of the Student Research Committee of the university, including 38 medical students and 15 health students, who were selected through census sampling due to their limited number. The data were selected with 2 questionnaires: demographic characteristics and biostatistics multiple-choice questions to assess learning of statistical concepts in three levels of literacy, reasoning, and thinking. The validity and reliability of the translated statistics questions were assessed through content validity and bisection, respectively. The data were analyzed in SPSS-16 through independent and paired t-tests and analysis of variance.
Results: The results showed a statistical difference between the groups in both faculties in statistical literacy and reasoning in blended learning (p>0.05). However, regarding statistical thinking, the difference was significant in the health faculty (p=0.044) and insignificant in the medical school (p>0.05).
Conclusion: Given the statistical difference observed in the health group, we suggest holding in-person courses at first and online courses then.
Keywords: Biostatistics, Censuses, Literacy, Reproducibility of results, Students, Medical Universities

Background
Fundamental changes in human societies, especially in education, are much more than turning chalk into a marker pen or a blackboard into a whiteboard. The new era requires employees who are ready to get lifelong education. Intel’s CEO states: “We do not need employees who want to work for us for 40 years based on only 4 years of university education; we need employees who can continuously learn and scientifically improve in 40 years” (1). Educational problems such as information explosion, a significant increase in the number of learners who want to improve their knowledge and progress in life, improper use of resources (human resources, educational equipment, or educational space), and outdated teaching methods require more in-depth attention. Enjoying educational technology and new teaching methods is one of the solutions (2). Educational technology has developed through three stages over the past years; group education, individual education, and education in small groups. Group education seems to lack the necessary efficiency in learning learners due to the possible inactivity of the learners during education. The second stage of development, i.e., individual education, dating back to 1960s, is emphasized in educational technology in which it is always tried to develop educational applications based on the stimulus-response technique; as a result, educational tools provide rapid responses to learners. The small group teaching/learning system, emphasized currently, is the third stage of educational technology. According to the teachings of this stage, education should provide conditions for active participation and collaboration of the teacher and learners in the education in small groups (3,4).
The teaching method is the learning key for students (5). According to Harvey and Vaughan, a strong relationship exists between how individuals learn and how they answer the situations (6). Studies show that learning occurs by seeing (82%), hearing (11%), touching (1.5%), tasting (1%), and smelling (2.5%) (7). Teaching is performed directly through lectures, live performances, role-playing, practical work, and discussion and indirectly through movies, objects, books, and booklets (8); and now teaching with a computer, either with CD or distance learning, can be added to this list (9). Meanwhile, web-based learning has attracted attention by providing user-friendly features such as, anytime and anywhere, ease of finding, ease of understanding, self-efficacy, need-based learning, and independent learning based on interests and talents (10,11). The blended learning approach is currently accepted in educational planning and seeks proper ways to combine media, aiming to effectively support learners, either individually or in a group, through official or nonofficial methods (12). A logical arrangement of in-person and online courses is necessary to successfully achieve the education objectives in blended learning of medical fields (13,14). Numerous studies performed in various areas, especially in web-based education, have confirmed the positive effect of this method on learners in comparison with traditional learning (15-17). In addition, most review articles suggest the simultaneous use of e-learning and other educational methods (blended learning) to increase the learning level of learners (2,12,18,19). Undoubtedly, these studies indicate the importance and strength of blended learning; however, the role of sequence in blended learning as an important issue is less considered (20). Although we are aware of the effectiveness of combined electronic and traditional education, the sequence of these methods should be identified, and this study aims to find the answer. On the other hand, biostatistics is an increasingly used field in all research areas, from industry, agriculture, economics, and business to health, biology, biotechnology, and medicine (21). It is also a prerequisite course for all medical sciences students willing to perform research. Given the importance of the sequence in educational planning, this comparative study was carried out to evaluate the effect of blended web-based learning and workshop learning on the learning level of biostatistics in the cognitive area, considering the Rumsey, Garfield, and Chance classification in statistics learning and assessment (22), in students who were members of the Research Committee of the Arak University of Medical Sciences. 

Materials and Methods
Based on the objective, this is a fundamental study of quasi-experimental type and alternative treatment design. The study population included all the students who were members of the Student Research Committee of the Arak University of Medical Sciences and were willing to participate in the study. Out of 26 health faculty students, 11 were excluded due to concurrency of unpredicted extra obligatory classes, and out of 41 medical school students, 3 were excluded due to absence from the tests. The remaining students were randomly divided into two equal groups (Table 1). The data were analyzed in SPSS 16 (SPSS Inc, Chicago, USA) through independent and paired t-tests and analysis of variance.

Study environment
The study was performed at Arak University of Medical Sciences. The in-person workshop was held in the meeting hall, and the Internet was used for web-based education.

Instruments
The data were collected with a questionnaire including the demographic characteristics and 30 multiple-choice questions to assess the learning level of biostatistics. The instrument was a Farsi translation of a standard questionnaire called Comprehensive Assessment of Outcomes in Statistics (CAOS) as the result of the Assessment Resource Tools for Improving Statistical Thinking (ARTIST) project. This instrument was developed by Garfield and Gall in 1999 to evaluate the assessment challenges in the education of statistics and was funded by the National Science Foundation (NSF). The ARTIST website currently provides an extensive type of assessment resources for the evaluation of students’ statistical literacy (such as understanding words and signs, ability to read and interpret diagrams, and terminology), statistical reasoning (such as reasoning with statistical data), and statistical thinking (such as questioning and decision-making related to statistical data). These resources are designed to assist the faculty members and instructors teaching statistics in different majors (such as mathematics, statistics, and psychology) to assess the learning of statistics in students (23). Regarding the content of the questions, the chapters of this questionnaire include data collection and design (Chapter 1), graphical representations (Chapter 2), variability (Chapter 3), sampling variability (Chapter 4), tests of significance (Chapter 5), and bivariate data (Chapter 6).
The content validity of the statistics exam questions (CAOS) has been evaluated in two huge assessments performed in 2004 and 2006 (23). The questionnaire’s items were translated into Farsi and sent to 3 experts for evaluating their content validity and consistency with the original questions: finally, 30 items were selected out of the total 40 questions.
The reliability of the original CAOS questions has been identified based on Cronbach’s alpha (0.77) (24). We used the bisection method to evaluate the Farsi version. According to the Guttman bisection scale, the final reliability of the questionnaire was 0.685, showing moderate, acceptable reliability (25). In addition, Cronbach’s alpha was calculated for all items (0.714). The test and the learners learning assessment based on different learning levels were scored according to emails with Robert delMas and Joan Garfield, the principal researchers of ARTIST (Table 2).
The mean score of each group of items constituted a single score and was used to compare the groups in each faculty through the t-test. In addition, the paired t-test was utilized to evaluate changes in the learning level of each group as a pretest-posttest. Finally, the total score of the three levels of statistical literacy, statistical reasoning, and statistical thinking in the groups of each faculty was compared using the independent t-test. It should be noted that the pretest scores of the groups in each faculty were analyzed to evaluate the effect of simple random division on their equality. The reliability of other questionnaires was assessed with SPSS 16.

Inclusion and exclusion criteria
Inclusion criteria were a bachelor or higher degree student, a member of the Student Research Committee of the Arak University of Medical Sciences, at least one semester passed in the relevant major, no withdrawal student, or graduated in another major than the present one. The exclusion criteria were incomplete participation in the educational courses, absence in the tests, and having the biostatistics course in the ongoing semester.

Implementation method
The total number of eligible people registered was 26 students in the health faculty and 41 students in the medical school. Out of 26 health faculty students, 11 were excluded due to concurrency of unpredicted extra classes, and out of 41 medical school students, 3 were excluded due to absence from the tests. The participants were randomly divided into two groups of 8 and 7 in the health faculty and two groups of 17 and 21 in the medical school. An educational session was held before the beginning of the study to introduce the web-based educational environment to the groups. In this session, a written consent form was also obtained from students and a pretest of all statistic questions was given. The students in both groups received an education based on the traditional (in-person) workshop learning approach (first group), the web-based e-learning approach (second group) in the first session, web-based e-learning approach (first group), and the traditional workshop learning approach (second group) in the second session. The two sessions had a 5-day interval, and the workshop (in-person) and web-based (online) learning were held at the same time for 2 hrs in two separate halls in the building of Arak University of Medical Sciences. Immediately after each session, an exam was given on the questions of that session. The educational content was separately prepared by two professors in the faculties. The site features, the method of presentation of subjects, the method of communication with online students, and paying attention to the sent messages were taught to professors in a private session. In total, four workshops were held in both faculties; two for the health faculty and two for the medical school. We managed the site learning environment with the OpenMeetings open-source software. To test the software abilities, we conducted a pilot educational course in two 2-hr sessions in the Payam Noor faculty in Khomein city for teaching Excel. The results of this pilot revealed that showing the professor’s image causes frequently interrupted the connection of students with the website due to weak telecommunication infrastructures; therefore, during the main study, the users only could hear the professor’s voice. In this study, we compared the sequence of blended learning using the alternative treatment design. The counterbalanced design is a method for determining the sequence of interventions in an experimental or quasi-experimental study. In the first stage of this method, with only two interventions of A and B, the researcher examines the volunteers with both interventions; thus, divides them into two groups. One group receives intervention A after intervention B, and the other group receives intervention B after intervention A. This type of counterbalanced design which consists of only two interventions of A and B is called the alternative treatment design (Figure 1) (26).

Ethical considerations
Ethical considerations consisted of stating the research objective to the officials of the Student Research Committee, the confidentiality of the characteristics and the evaluation results, honesty in all stages of the study such as from completing the questionnaire to data analysis and expression of results, and mentioning the references used in the study.

Results
The mean age of students in the medical school with MD degree was 22.72±0.895 years in women and 24.25±4.833 years in men and the mean age of students all of whom were female in the health faculty with bachelor’s degrees was 21.33±0.9 years. The highest mean age (24.1±4.78 years) pertained to the first group in the medical school and the lowest mean age (21.13±0.84 years) pertained to the first group in the health faculty.
Students participating in the study were in the fifth semester or higher. The number of singles in both faculties and all groups (86.8%) was higher than married students (13.2%). Half of the participants (50%) in both faculties resided in dormitories. In the medical school, more than half of the students in the first group (52.4%) and most of the students in the first group (52.9%) (Table 3) stated that they were moderately and highly familiar with Windows, respectively. However, in the health faculty, the majority of students in both groups asserted that they were moderately familiar with Windows. In both faculties and all groups, most students stated their moderate familiarity with Windows. We used Leven’s test (P=0.626), and equality of variances in both groups (variance of the first group = 1.841 and variance of the second group = 1.328) to evaluate the correctness of the random division of the students into the groups. No significant difference was observed between the mean total scores of the first and second pretests in the first group (3.9) and the second group (4.53) according to the equal variance t-test formula (t=-1.172, df=36, p=0.249); therefore, random division of the participants for equivalency of the two groups was successful.

Table 1. Frequency of the students based on the group and major

Table 2. Assessment method of learning levels based on items number and test number

Table 3. Frequency distribution of the students in terms of faculty, group number, and sex

Table 4. The adjusted and unadjusted mean of the statistical thinking level and the posttest variability using the pretest as the covariate

Table 5. ANCOVA for evaluating the difference in the means of posttest considering the pretest as the covariate -Tests of between-subjects effects

1. R Squared = 0/301 (Adjusted R Squared = 0/184).

Table 6. Results of the Sample Group Statistics and the independent t-test using the pretest and posttest data of the learning index in the medical school and health faculty

Table 7. Results of the Paired Samples Statistics and the paired t-test using the pretest and posttest data of the learning index in the medical and health faculties

Calculation of the learning index
To calculate the learning index, the scores obtained from the level of statistical literacy, statistical reasoning, and statistical thinking were summed and the groups were compared in each faculty (Table 4). 
Regarding the statistical thinking level in the health faculty, significant differences existed in the mean scores of the posttest between the first and second groups (F=5, df=1.12, p<0.05) (Table 5).
Regarding the statistical literacy level and statistical reasoning level as well as the difference in the mean cumulative score of the posttest between the two groups in the medical school (t=-1.24, df=36, p=0.222) and the health faculty (t=1.24, df=13, p=0.236), the results showed no difference between the first and second groups in both faculties in terms of the learning index (Table 6).
In addition, the correlation t-test (paired t-test) was performed to find the occurred change in each group by comparing the means of the pretest and posttest of each group separately (Table 7).
According to the results of the test obtained based on the accumulation of the scores of three levels of statistical literacy, reasoning, and thinking at two temporal times of before education and after education, the table of paired samples test demonstrates that the difference in the means in both faculties and all 4 groups was significant (p<0.05), indicating the effect of education on the biostatistics learning index among students (Table 7).

Discussion
As a novel experiment, this study aimed to evaluate the sequence of executing a 2-day blended learning workshop in the medical school and health faculty of Arak University of Medical Sciences. We compared separately the learning indices of statistical literacy, reasoning, and thinking in two groups in these faculties. The results of the study showed no significant difference between the groups of both faculties in the statistical literacy level of students who received the workshop education first and then the web-based education (A), and students who received the web-based education first and then the workshop education (B) (p>0.05), meaning that at this level of learning, none of the AB and BA sequences were superior to each other. However, according to the paired t-test results, a significant difference existed in the means of the pretest (3.9) and posttest (5.5) of the first group compared to the means of the pretest (4.5) and posttest (5.4) of the second group in the medical school. Therefore, learning was improved more in the first group that received the in-person education first and then the web-based education.
Studies in this regard show that students have problems understanding concepts related to statistical changes such as assessment of variability, sample diversity, and sampling distributions. Comparison of this level of statistical learning with Bloom’s taxonomy cognitive levels indicates that the former includes most of the elements of the upper three levels of the latter (23), and achieving it requires further effort. Analysis of the data of this learning level demonstrated that since the results of the two faculties were dissimilar and the sample size was lower in the health faculty, the observed difference cannot confirm the superiority of the AB method to the BA method by rejecting hypothesis 3. However, this superiority can be somehow justified according to the mentioned issues and the results shown at these levels. Statistical literacy emphasizes the basic concepts of statistics, such as understanding the statistical terminology and signs and the ability to read and interpret diagrams. In Bloom’s taxonomy of cognitive levels, this level includes the knowledge level. Weakness in this section may result in difficult learning of the upcoming subjects by learners. Studies have shown that students have problems with expressing distributions and presenting distributions graphically. We suggest holding blended learning workshops through any of the two methods (23). Regarding the statistical reasoning level, a comparison of the pretest and posttest means in each group showed no significant difference in the faculties (the medical school: the first group=0.008 and the second group=0.014; the health faculty: the first group=0.100 and the second group=0.029); however, improvement in the first group was better than the second group. Statistical reasoning is a method through which people are reasoning with statistical ideas and specifying the meaning of statistical data. Statistical reasoning may be associated with the construction of a concept with another concept (for example, central index and dispersion index) or combining ideas about data and possibilities. The problems of students have been well documented by understanding the possibility and inference in the odds of events. This level is equal to the understanding level and a part of the analysis level of Bloom’s cognitive level (23). We suggest holding blended learning sessions through any of the two methods. No significant difference existed in the statistical thinking level in the medical school, whereas a difference was observed in the health faculty after performing amendments. 

Conclusion
The results of this study showed no significant difference in the groups of the medical school, but a significant difference existed in the health faculty between the adjusted means of the first group (5.1) and the second group (3.5) (p=0.044), meaning that the education sequence of AB was better than BA. Statistical thinking refers to the cognition and understanding of the whole research process (from designing of the questions to the collection of the data, selection of analysis methods, testing of assumptions, so on), understanding the models used for sampling of random phenomena, understanding how the data are produced for estimation of probability, cognition of how, when, and why the inference tool can be used for this reason, and ability to understand and apply the problem background for designing and evaluation of analyses and plotting of the results. We suggest holding blended learning workshops through any of the two performed methods, with the priority of the in-person to the web-based sessions. There are a limited number of studies regarding the evaluation of blended learning sequence; as a result, due to the high costs of education (27,28), performing such studies can not only optimize the cost resources but also can facilitate the selection of proper education methods.

Recommendations
Researches have been carried out regarding the effectiveness of Blended Learning in various fields, and most of them have confirmed the suitability of this method, but what has been neglected and less looked at is the discussion of the merits and demerits of face-to-face and non-face-to-face teaching methods. Definitely, according to this issue, it is possible to save money and improve the quality of learning in students.
Examining the merits and demerits of combined teaching methods in courses that have a theoretical and practical unit. 
Examining the interest (according to the three factors of attitude, motivation and satisfaction) and its relationship with the sequence in training courses.

Study limitations
The limited number of references and backgrounds due to the novelty of this study in the medical sciences universities, the difficulty of coordinating the implementation of the workshop due to the need for audio-visual equipment and computer, and the lack of proper internet bandwidth were among the limitations of the present study.

Ethics approval and consent to participate
This study was approved by the Ethical Board of Arak University of Medical Sciences. Verbal informed consent was obtained from all the participants. All the methods were carried out in accordance with relevant guidelines and regulations. The privacy of the participants was well protected. The participants were voluntary, with consent at the beginning of the questionnaires. Only participants who signed the informed consent document could complete the questionnaires. The questionnaires did not include names and personal identification information. The registration number of this project in Arak University of Medical Sciences is 24. Due to the fact that the present study was not a clinical trial and drug interactions, the university did not recognize the need for a code of ethics and it was enough to comply with ethical considerations and written informed consent from the students. 

Consent for publication
Not applicable.

Acknowledgements
The authors wish to sincerely thank all the students who participated in the study and our friends Dr. Majid Taheri, Mehdi Abedi and dear professor Dr.Bob delMas and Dr. Joan Garfield.

Conflict of Interest
The authors declare that they have no competing interests.

Funding
Not applicable.