Assessing the Performance of the Health, Safety and Environment Management System (HSE) using the Modified Balanced Scorecard Model

Introduction

Due to sensitive environmental and social issues at the present time, modern organizations need to pay extra attention to the protection of the environment and the health and well-being of the employees in addition to the importance of customer’s satisfaction. The most important issue that encourages the companies and institutions to use safety, health, and environmental systems is the emergence of a specific level of demands and expectations from stakeholders in the fields of safety, health, and environment. However, organizations go beyond the rules and standards as they are supposed to be the preferred organizations in that area. This also strengthens its credibility and reputation for investors, customers, and stakeholders, who are sensitive to the issues by such as increase in productivity, reduction in environmental risks and promoting the health and safety ¹. The process of evaluation refers to the process of determining the importance of an activity, policy or program. In other words, the evaluation system, the intervention process of the objectives, and the plans are systematically designed ² .

The evaluation of performance is a continuous process that measures the degree of deviation from the goals. In this process, the efficiency of the used resources, the work processes, the quality of product (output processes), and services are tested ³. Since one of the most important and critical factors in achieving the goals of an organization in terms of Health, Safety, and Environment (HSE) is the quality and effectiveness of safety, health, and environmental management systems, then the organization or the industry needs to evaluate the performance of above mentioned systems for determination of effectiveness ^4,5 . The significance of an HSE management system is determined by the number of accidents and their costs to the system. For example, according to data from the International Safety Association of the United States, there are about 2,200 deaths and 220,000 injuries per year caused by the occupational accidents, which impose significant costs ⁶. Therefore, it can be concluded that the HSE management system must be continuously evaluated to improve the level of HSE management activities as well as to reduce work accidents and human casualties. Many organizations use simple indicators to assess their core areas of work, but health and safety professionals do not have such indicators ⁷.

The balanced scorecard with its flexibility feature can adapt itself to the specific features of the safety and health management system, so it will be a suitable and cost effective tool to combine the different indicators and reflect the interests of all stakeholders in the operational plans of an organization for the occupational safety and health^6,7. The Balanced Scorecard approach, first introduced by Kaplan and Norton in 1992, has been then used by many organizations and researchers as a tool for assessing the performance of the systems. There are usually four indicators to evaluate the performance in a balanced scorecard model used: financial performance, customer focus, internal processes, and growth and learning ⁸. The HSE management systems in each organization or industry focus on three sectors including human, capital, and environment ⁹. The reasons for this are the interest of most organizations and industries in maintaining their assets, such as labor, preventing the loss of funds, maintaining the intangible assets of an organization or industry including culture, internal and external relations; being competitive ⁹. On the other hand, looking at the environment is not similar to that looking at capital and human beings. Some high-income organizations and industries do not pay attention to the problems produced by themselves to the environment and, in other words, the most important issue for such organizations is earning more income. All organizations are responsible for the environment; however, it is hard to put pressure on them by issuing fines as they prefer to pay fines instead of paying attention to the environment. Due to the resilience of the balanced scorecard model, and also the importance of the environmental review for the reasons mentioned, in this study, a separate layer was devoted to the environmental dimension of the HSE management system ⁹.
With regard to the mentioned issues, the purpose of this study was to evaluate the performance of the HSE management system of Saba Tire Cord Company in Zanjan using a balanced scorecard model.

Materials and Methods

In this study, which took place at Saba Tire Cord Company in Zanjan in 2016, the performance of the health and safety management system was evaluated using a modified balanced scorecard model. Since the balanced scorecard model did not have a standard questionnaire in the field of HSE management, we designed a questionnaire in that regard. Designing such a questionnaire based on the needs and models and the evaluation of standards in the field of HSE management performance, we also used five recommended criteria. The questionnaire as a tool should be tested in terms of validity and reliability. Usually the content validity is used to determine the validity of questionnaires like this one, especially questionnaires which are designed for a specific work place. Since the purpose of this study was to design a specific questionnaire for a particular work environment, we used the content validity. There are different methods to determine the reliability of a questionnaire depending on the function and nature of that questionnaire, so we preferred the Cronbach alpha method. This method is the most common method for determining the internal consistency and for representing the degree of proportionality of a group of items that measure a structure. The minimum acceptable coefficient for reliability is 0.7. In our study and every study that uses the Likert scale in the questionnaire instrument, is under investigation this coefficient is necessary to assess the reliability of the internal consistency. In 2003, the alpha value of 0.9, and more excellent, while 0.89-0.8 was good and 0.7 - .8 was acceptable, 0.6-0.7, was considered weak and less than 0.5 was unacceptable ¹⁰. As mentioned, the content validity has been used to assess the validity of this questionnaire and can be tested quantitatively and qualitatively. To evaluate the content validity of questionnaire, from point of views of a statistician, an epidemiologist and interviewing specialists were used, according to their ideas, the proper words, the importance of items, and the placement of items in their proper place in a correct grammar were modified ¹¹. The Content Validity Index (CVI) were used for quantitative content validity.

In order to calculate the content validity index in this questionnaire, 19 experts’ opinions were used in two groups of 10 and 9 with different tendencies. Therefore, we distributed the questions separately among the ten professors and experts. Other questions were designed in four other dimensions of the balanced scorecard (growth and learning, finance, customer, and internal processes) and sent to 9 of the (state-recognized) expertise in the field who had education and specialties in the same field. The questionnaires were evaluated using three spectra: “the spectrum of necessary, the spectrum of useful but not necessary and unnecessary”. Then, the Lawsche formula was used to calculate the CVI.

The calculated ratios for each question must be compared with the numbers presented in the table‌; then, if the value obtained from the table is greater than or equal to the numbers then the validity of that question is verified in terms of content validity ^11,12.

After determining and calculating the Content Validity Ratio (CVR), the CVI index can be calculated. To compute this index, the selected professionals should answer each question in the initial questionnaire based on the three criteria of simplicity, relevance and clarity regarding the 4-part Likert scale. The CVI formula was used to calculate the content validity index. The formula for calculating the CVI is as follows: Content Validity Index = (total score for each item with ratings 3 and 4) / total number of responses. The acceptance of each item was based on the following criterion ¹³: Content Validity Score “above 0.79” as appropriate, content validity index score “0.7 to 0.79” as questionable and needs to be amended and any less than 0.7” as unacceptable. The last one was removed from the study.

When the validity and reliability of the questionnaire were obtained, the data collection was started. The population of this research included all personnel of Saba Tire Cord Company in Zanjan city. The total statistical population, or the number of personnel in the company, was 552. To survey the workers, a proportional stratified sampling based on the company units was carried out. There was no presupposition to calculate the sample size, so a standard deviation of 100 and a precision of 10 was suggested. Therefore, the number of samples reached a figure of 384. Considering the number of 552 workers in the study, we used the sample size correction coefficient for a limited community and the final sample size came up to 290 and got rounded to 300 for ease of result analysis. We used the SPSS 11.5 software.

Table 1. The study structure for HSE management system performance

Table 2. Distance scale

To identify the weaknesses and strengths of the listed HSE management system in more details, the variables were extracted from the original questionnaire. The use of these variables was to design two paired and pop-up questionnaires. There were 5 criteria plus 22 sub-criteria in the paired questionnaire (Table 1) and 22 sub-criteria, the three options (safety, health and environment) were used in design of the Topsis Questionnaire. Various methods have been utilized for selecting the criteria to evaluate the designed questionnaire. For example, Keeney et al and Saaty et al presented some methods ^14,15 that we included. Erwin et al recommended a maximum of 2±7 criteria per branch ¹⁶. In the present study, as a modified balanced scorecard model, five criteria were selected (Table 1). Evaluation of relative importance of options is one of the most important and difficult decision-making processes that any mistake in it can cause significant uncertainty in the final decision ¹⁷. Therefore, in a decision-making process, it is an error to put decisions on the basis of mental judgment and use it directly to calculate the superior option. There are several methods that can fairly calculate the accurate weight of indices or options. Distance scale is a general method for measuring a qualitative index with a bipolar distance scale (Table 2).

After setting criteria and the constraints affecting the decision making processes, the matrix of the paired comparison matrices was provided to the statistical society in which 15 experts of Saba Tire Cord Company were selected from different units. The data were entered into the Expert Choice software and the pairwise comparison of the criteria and the calculation of the incompatibility rate were carried out. For this purpose, the hierarchical structure was used in accordance with the following diagram (Diagram 1).

This structure consists of four levels of purpose (HSE performance management assessment), criteria (five scorecard scales), sub-criteria (22 sub-criteria), and options (safety, health and environment). As noted above, in this study, the three safety, health and environmental choices were ranked based on the existing conditions. Therefore, a separate questionnaire was designed in which three “health, safety, and environmental” measures were assessed by 22 sub criteria of AHP questionnaire. The questionnaire was distributed among 15 experts of Saba Tire Cord Company. The data were ranked using the TOPSIS software. In order to weigh the options in this questionnaire, the intervals of 1 for very weak, 2 for weak, 3 for moderate, 4 for good, 5 for very good were used.

Results

In this study, the performance of the HSE management system was tested from two points of personnel’s’ and experts’ views. The results are presented in tables 3 and 4.

The experts’ point of view and the extracted weights by AHP & TOPSIS are shown in table 4.

Table 3. The performance status of BSC dimension reported by personnel

Table 4: The obtained weight of the layers and substrates using the Expert Choice software

The incompatibility rates in different dimensions of the balanced scorecard were calculated for environment, internal process, financial, customers, growth and learnings as: 0.03, 0.09, 0.03, 0.08 and 0.05 respectively.

One of the important advantages of the hierarchical analysis process is to measure and control the compatibility of each matrix and decision. The acceptable range of inconsistency in each system depends on the decision maker, but in general, Saaty et al ¹⁵ suggest that if the decision inconsistency is greater than 0.1, then it is better for the decision maker to revise his judgments. According our results, the acceptability and meaningfulness of each dimension of a balanced scorecard is proven. Chart 1 summarizes the results of data analysis as well as the ranking of options using TOPSIS software.

Discussion

This study was conducted in the framework of a balanced scorecard model in which the AHP, and TOPSIS methods were simultaneously used to determine the importance of criteria and sub-criteria, and then the ranking of options was done. We found that the performance of the HSE management system from a customers’ perspective is better than other dimensions.
Also, for ranking options to select the best option in terms of performance of the HSE management system, the safety option showed the best status of the performance of the HSE management system. Similar to ours in a study by Shafaei Gholami et al ¹⁸, the performance of the HSE system was evaluated using a scorecard model. Their study used the flexibility of the model and reduced the dimensions (merging two layers of customer and financial and transforming model of the scorecard model into a three-point model); these three views had 14 substrates ¹⁸. According to our analyses based on the scoring method of Deming cycle (PDCA), the HSE management system of this organization (MAPNA company) has a good performance in all of the sub criteria and the performance of the HSE system in these sub-sectors including (organization and provision of resources for the successful management of the HSE system, the management of the main and secondary contractors in the field of HSE, planning, and designing ⁶. The necessary systems for the establishment of the principles of HSE, implementation and monitoring of designed systems HSE, and auditing and improving the performance criteria of the HSE, are within the acceptable level of strengths ⁷.
On the other hand, in this study, from the perspective of stakeholders and with regard to the sub-sectors, the value presented to customers, the value presented to society and the government, and human capital in the HSE field, the lowest points were observed and they are considered as weakness points. As far as we know, there are few studies that simultaneously examined the HSE management system from two personnel’s and experts’ points of view using a balanced scorecard ⁹. This study tried to use the views of all employees within the company for final decision making. Our study showed that the performance of the HSE system from personnel’s perspective was the best performance.

Our results showed that the financial dimension was the lowest criteria in terms of the HSE management system auditory attention. We, for the time, showed that we used two hierarchical analysis and the TOPSIS analysis methods together to evaluate the performance of the HSE management system. For this purpose, two expert Choice and TOPSIS software were used. Based on the five main criteria, the final weights of the criteria and sub-criteria were determined from the experts’ point of view. Our results showed that, the customer benchmark had the best performance in the HSE management system from the point view of expertise. Followed by the financial layer, the internal processes layer, the growth and learning layer, and the environment in that order. Also in the sub-criteria based on the final weight obtained from the sub-criteria, the sub-criterion for maintenance and promotion of the health and safety of the workforce showed the highest weight from the experts’ point of view and also the criteria of documenting and controlling documentation were of the lowest weight among all sub-criteria. Hence, it can be discussed that due to the low value of the environmental criterion, the environmental issues are less respected, and that can lead to environmental problems, such as environmental pollution ¹⁶. Therefore, the company must follow the environmental requirements seriously. Given the weight by the customer layer, it can be discussed that the HSE management system has paid a lot of attention to work and labor issues by enforcing legal requirements and increasing worker safety in the workplace. Consequently, that policy significantly reduces the incidents making people happier in work places regarding the sub-standards of the environmental layer. Considering the weight of our results, it is possible to say that improvement of the performance of the HSE management system is possible by putting the environmental conservation issues in company’s policies and practices. Internal processes layer, in order to have a better performance from the HSE management, the system and its agents must convince the senior managers to conduct the audits, as well as setting benchmarks for evaluation and improvement ¹⁷. However, it has not yet been possible to make an effective decision on the provision of raw materials; in the customer layer, the HSE management system has a special focus on labor’s safety, which can be considered as an advantage and also a disadvantage. An advantage is considered in terms of maintaining both the human resources and financial resources and the reputation of the organization, however on the other hand, by too much attention to this case, other sub-layers are remained unattended ¹⁷. In the growth and learning layer, the documentation and documentation control with a relative weight of 0.157 is of less importance. The reason for this can be referred to the company’s policy. In other words, in the company’s policies and goals, training is seen and implemented tightly, but less attention is paid to documentation issues. This can also be compensated by periodically revisions of the policies ⁸.

Among the limitations of this study, one can point to a balanced scorecard which far the customer’s perspective, it includes both domestic and external customers, and the possibility of simultaneous examination of these customers (domestic and external) for reasons such as diversification of the products of the company. Thus, the variety of external customers, the timing of detection and finding them and the possibility of some not cooperating, the research team decided to abandon the assessment of the performance of the HSE management system by external customers. Therefore, for future studies, it is suggested that the layers of stakeholders be assessed from the perspective of both internal and external customers’ points of view. Also, considering the flexibility of the scorecard model, health and safety can be used as a perspective in reassessment to compare the results.

Conclusion

Regarding the results, it can be concluded that the HSE management system was very good from the customer’s point of view, from the perspective of the personnel’s and from the perspective of the experts. The performance of the HSE system in terms of underlining aspect of the workforce safety was placed higher and looked more important. We believe that the results from the TOPSIS software also offer a better safety rating than health and the environment.

Acknowledgments

We must say “thank you” to the Deputy of Research and Technology of Zanjan University of Medical Sciences for the financial support, and also to the staff of Saba Tire Cord Company in Zanjan and to all who have helped us to carry out this research.