Cultural Adaption, Validation and Factor Analysis of the Persian Version of the Non-Nutritive Sucking Scoring System

Document Type : Original article

Authors

1 Department of Speech and Language Pathology, Rehabilitation Research Center, School of Rehabilitation Sciences, Iran University of Medical Sciences, Tehran, Iran

2 Shahid Akbarabadi Clinical Research Development Unit (ShACRDU), Iran University of Medical Sciences, Tehran, Iran

Abstract

Abstract
Background: Valid and reliable tools are required to measure non-nutritive sucking and oral feeding readiness of preterm infants. The aim of this study was to translate the Non-Nutritive Sucking scoring system (NNS) into Persian language and to evaluate its measurement properties.
Methods: A cross-sectional study was conducted to translate and cross-culturally adapt the NNS to Persian language (P-NNS) following steps described in guidelines. Eighty-eight preterm infants participated in this study. Psychometric properties of floor/ceiling effects, internal consistency reliability, inter-rater reliability, Standard Error of Measurement (SEM), Smallest Detectable Change (SDC) and construct validity were tested. Factor analysis was performed to determine the P-NNS structure.
Results: There were no floor or ceiling effects that indicate the content and responsiveness of P-NNS. Internal consistency was high (Cronbach’s α 0.905). Item–total correlations exceeded acceptable standard of 0.3 for the all the items (0.3–0.9). The inter-rater reliability was excellent (k=0.91, SE=0.04; p<0.001). SEM and SDC were 8.04 and 22.28, respectively. Construct validity was supported by a significant correlation between the P-NNS total score and the Persian version of the Early Feeding Skills (EFS) total score (r=0.94, p<0.001). Explanatory factor analysis revealed 2 components for p-NNS.
Conclusion: The P-NNS was cross-culturally adapted to Persian and demonstrated to be a reliable and valid instrument to measure non-nutritive sucking and oral feeding readiness of preterm infants.

Keywords

Main Subjects


Abstract
Background: Valid and reliable tools are required to measure non-nutritive sucking and oral feeding readiness of preterm infants. The aim of this study was to translate the Non-Nutritive Sucking scoring system (NNS) into Persian language and to evaluate its measurement properties.
Methods: A cross-sectional study was conducted to translate and cross-culturally adapt the NNS to Persian language (P-NNS) following steps described in guidelines. Eighty-eight preterm infants participated in this study. Psychometric properties of floor/ceiling effects, internal consistency reliability, inter-rater reliability, Standard Error of Measurement (SEM), Smallest Detectable Change (SDC) and construct validity were tested. Factor analysis was performed to determine the P-NNS structure.
Results: There were no floor or ceiling effects that indicate the content and responsiveness of P-NNS. Internal consistency was high (Cronbach’s α 0.905). Item–total correlations exceeded acceptable standard of 0.3 for the all the items (0.3–0.9). The inter-rater reliability was excellent (k=0.91, SE=0.04; p<0.001). SEM and SDC were 8.04 and 22.28, respectively. Construct validity was supported by a significant correlation between the P-NNS total score and the Persian version of the Early Feeding Skills (EFS) total score (r=0.94, p<0.001). Explanatory factor analysis revealed 2 components for p-NNS.
Conclusion: The P-NNS was cross-culturally adapted to Persian and demonstrated to be a reliable and valid instrument to measure non-nutritive sucking and oral feeding readiness of preterm infants.
Keywords: Cross-Sectional Studies, Factor Analysis, Infant,Newborn, Persian people, Premature, Psychometrics, Reproducibility of results, Statistical

Introduction
Feeding problems defined as any disturbance of oral feeding is a major issue commonly experienced by premature infants (1-3). The prevalence of feeding problems was reported 50% to 80% in premature infants (4,5). Feeding problems in premature infants can lead to malnutrition, developmental abnormalities, and cognitive dysfunction (6-8). Moreover, it has a negative impact on various economic and social dimensions of the family (9-11).
Successful oral feeding requires adequate oral sensorimotor performance, sucking, swallowing skills, and coordination of sucking, swallowing and breathing (12,13). Sucking as a critical feeding skill indicates the safety of beginning oral feeding (1,2,14). Sucking is considered into two situations including nutritive and non-nutritive (3,5) and as a crucial oral reflex is necessary for oral feeding and self-regulation in infants (4). Therefore, it should be considered in the assessment of the oral intake readiness and treatment of feeding problems in preterm infants (4,11,15,16). Reliable and valid measurements are essential for assessing sucking in preterm infants. There are a few clinical tools to evaluate sucking in preterm infants such as oral motor evaluation scale of newborn (NOMAS) and the Non-Nutritional Sucking scoring system (NNS) (17,18). The NNS is a valid and reliable tool that developed for evaluation of the sucking skills to specify the safe beginning of oral feeding (17,19). Moreover, NNS is a sucking evaluation protocol that objectively scored the criteria necessary for oral feeding in preterm infants (17). The original version of NNS is in English (17). Moreover, the NNS has been translated in Chinese and used as a reliable and valid tool for evaluating oral motor function of premature infants in China (20).
There is a need for clinical assessment tools available in the Persian language to be used for the assessment of sucking skills in preterm infants. The NNS has not been translated and culturally adapted into Persian language. Therefore, the goal of this study is to translate, culturally adapt the Persian version of the NNS and to assess validity and reliability of the Persian NNS (P-NNS) in the preterm infants.

Materials and Methods
This cross-sectional study was approved by the Ethical Committee of the Iran University of Medical Sciences (ethics code: IR.IUMS.REC.1399.1430).The procedure of study was completely explained to each mother’s infant, and written informed consent for participation in the study was obtained.

Participants
The preterm infants were enrolled from the Neonatal Intensive Care Unit (NICU) of the Akbarabadi Hospital of the Iran University of Medical Sciences (IUMS) in Tehran, Iran, from May to August 2021. The inclusion criteria were: (1) Gestational Age (GA) ≤36 weeks, (2) chronological age ≥2 days, (3) Apgar score ≥6 at five minutes after birth, (4) enteral feeding (either human milk or formula) by orogastric or nasogastric tube or orally, and (5) physiological stability. The presence of facial abnormalities, con-genital malformations, neurological disorders, and infections were considered as the exclusion criteria.

Instruments
The NNS scoring system is a 12-item clinical instrument for evaluation of non-nutritive sucking abilities at the beginning of oral feeding in preterm infants. A total NNS scoring scale between 50 and 86 indicates the infant’s ability to begin oral feeding. The NNS scoring scale is done by gloved finger of the Speech and Language Pathologist’s (SLP) right hand. At first, the perioral region specifically near the angle of the mouth is stimulated for evaluation of the rooting reflex. Then, the anterior the palate or lower gum and the tip of the tongue are touched for triggering the sucking reflex (17). 
The NNS scoring scale should be accomplished when the preterm infant is awake and has no symptoms of stress such as crying, choking, and eye blinking approximately 30 minutes before meal time (11).

Procedure
This study was conducted in two phases: (1) translation and adaption of the NNS questionnaire when permission was obtained from the developer of the original NNS, (2) investigation of the psychometric properties of the P-NNS. All the participants were assessed according to P-NNS and Persian version of Early Feeding Skills (EFS). The Persian EFS as a reliable and valid tool was applied for construct validity. The EFS is an assessment checklist for evaluation of oral feeding readiness in preterm infants. The EFS included four factors critical to successful oral feeding: remaining engaged in feeding, organizing oral motor function, coordinating swallowing and breathing, and maintaining physio-logical stability (21). For the inter-rater reliability, two trained SLPs independently scored the P-NNS for each preterm infant in a session. Inter-rater reliability was determined for the total scores and section scores. The raters were blinded to their ratings, and no discussion of the scores assigned was allowed.

Translation and cross-sectional adaption procedure
The translation and cross-cultural adaptation pro-cedure of the NNS to Persian language were performed following recommended guidelines (22, 23). Two professional translators with Persian as their mother language, one of them had medical background and was familiar with related concept, independently forward translated the original English version of the NNS to Persian language. After trans-lation, an expert committee together with both translators and a SLP who was involved in the study, discussed and synthesized a Persian version of the NNS questionnaire. The translated NNS was tested on a sample of 10 SLPs to obtain feedback on the acceptability and clarity of the items. Consequently, feedbacks and comments from all the professionals were reviewed and discussed with expert committee and Persian version of the NNS questionnaire was produced. Then, two bilingual translators, who were blinded to the study and had no prior knowledge of the instrument, back translated the preliminary version of the NNS to English. The expert committee developed the Persian version of the NNS (P-NNS) questionnaire (Appendix), considering the original English questionnaire, forward translations, consensus version of forward translations, and the back translations.

Statistical analysis
The SPSS v 11.5 (SPSS Inc; Chicago, IL) was applied to perform statistical analyses. Descriptive statistics was used to assess the missing data, distribution of the scores, and floor and ceiling effects for the total score (cut-off=15 %) (24). To determine the internal consistency reliability, the Cronbach’s α coefficient was used with a coefficient of 0.7 considered as acceptable (25). The Kappa coefficient was utilized to assess the level of agreement between the assessors. Evaluation criteria for kappa, using guidelines were as follows: Fair: 0.40 to 0.59; Good: 0.60 to 0.74; and Excellent: >0.74 (26,27). The Standard Error of Measurement (SEM) and Smallest Detectable Change (SDC) were calculated as SD √1˗ α and 1.96 × √2 × SEM, respectively. Spearman correlation coefficient was used to assess the construct validity of the P-NNS by relating this index to the Persian version of EFS. A correlation of 0.7 was considered acceptable for construct validity (24). The factor structure of the P-NNS was analyzed by Principal Component Analysis (PCA) with varimax rotation considering data suitability if Kaiser–Meyer–Olkin (KMO) >0.8 and a significant p-value <0.01 for Bartlett’s test of sphericity. A factor load coefficient of greater than 0.3 was considered for an item to be included in a factor. The factor loading of 0.3 is the minimum loading of an item in association with the underlying factor. The scree plot inflection, eigenvalue >1.0, and variance >10% were considered as a-priori requirements for factor extraction (28).

Results
Participants’ characteristics
A total of 88 preterm infants (male/female 46/42; mean gestational age 32.8±2.35 weeks; mean birth weight 1944.09±589.05 g) participated in this study. Thirty-three preterm infants (37.5%) were born with respiratory issues. The characteristics of infants are presented in table 1. 

 

Table 1. Demographic characteristics of the participants (n = 88)

 

Preterm infants

(n = 88)

Gestational age at birth (weeks)

32.8±2.35

Chronological age (weeks)

8.75±7.33

Apgar (median)

9

Birth weight (grams)

1944.09±589.050

Delivery mode

    Cesarean

    Vagina              

 

92%

8%

Breathing support

    Yes

    No

 

43.2

56.8

Translation
The process of translation of the P-NNS was performed without difficulty and the backward translation was corresponded to the original version. There was no difficulty in understanding the items and all of them were clear and relevant. There were no missing data, and all items of the questionnaire were responded.

Floor and ceiling effects
There were no missing data for individual items of P-NNS. The P-NNS scores ranged between 3 and 86 (50.12±26.07). No patients achieved minimum or maximum possible scores for the P-NNS. Descriptive statistics of mean and standard deviation for items, total scores, positive and negative items scores are summarized in table 2.

 

Table 2. Means and standard deviation (SD) for the items, total scores, and factors of persian NNS with standard error of measurement (SEM) and smallest detectable change (SDC) (n = 88)

Items

Mean

SD

SEM

(95% CI)

SDC

Item 1

3.09

1.68

0.52

1.44

Item 2

2.41

1.97

0.61

1.69

Item 3

8.33

3.86

1.19

3.30

Item 4

5.96

3.10

0.95

2.63

Item 5

5.56

3.03

0.93

2.57

Item 6

5.74

2.74

0.84

2.32

Item 7

8.86

4.78

1.47

4.07

Item 8

6.94

3.92

1.21

3.35

Item 9

6.88

4.02

1.24

3.43

Item 10

-0.78

0.86

0.27

0.74

Item 11

-0.48

0.71

0.22

0.61

Item 12

-2.44

3.27

1.01

2.80

Positive items score

53.78

24.58

7.58

21.01

Negative items score

-3.65

3.81

1.17

3.24

Total score

50.12

26.07

8.04

22.28

NNS: Non-Nutritive sucking scoring System; SD: Standard Deviation; SEM: Standard Error of Measurement; SDC: Smallest Detectable Change.

 

Internal consistency reliability 
The overall Cronbach’s α for 12-item P-NNS was 0.905. The Cronbach’s α for each of the item is shown 
in table 3. 

 

Table 3. Internal consistency reliability values for persian NNS (n = 88)

Items

Scale mean if item deleted

Scale variance if item deleted

Corrected item-total correlation

Squared multiple correlation

Cronbach’s alpha if item deleted

Item 1

46.98

619.207

0.61

0.64

0.91

Item 2

47.67

614.338

0.56

0.58

0.91

Item 3

41.75

525.431

0.75

0.72

0.891

Item 4

44.11

546.263

0.81

0.79

0.89

Item 5

44.52

541.241

0.87

0.86

0.88

Item 6

44.34

558.342

0.82

0.78

0.89

Item 7

41.21

462.562

0.91

0.89

0.88

Item 8

43.13

510.786

0.83

0.85

0.89

Item 9

43.19

504.319

0.84

0.82

0.88

Item 10

50.86

663.154

0.30

0.42

0.91

Item 11

50.55

658.778

0.37

0.38

0.91

Item 12

52.52

637.839

0.15

0.31

0.92

NNS: Non-Nutritive Sucking scoring system.

 

The SEM and SDC
Absolute reliability measures of the SEM and the SDC for 12-item P-NNS were 8.04 (CI 95%= ±0.59) and 22.28, respectively. The values of SEM and SDC for positive and negative items are shown in table 2.

Inter-rater Reliability
There was a significant agreement between two raters (p<0.001). Table 4 presents the results of the kappa for each item. The results demonstrated excellent agreement between raters for 11 (91%) items and good agreement for 1 (9%) item. The weighted κ values were 0.91, (SE=0.04, p<0.001) for total score.

Construct validity
The spearman correlation test revealed a significant relationship between the P-NNS and Persian version of the EFS scores (r=0.94, p<0.001). 

Factor analysis
The sampling adequacy was 0.84 as calculated with KMO. The Bartlett’s test of sphericity (Chi-square=853, df=87, p<0.001) showed that the correlation matrix was suitable for factor analysis. Principal component analysis extracted 2 latent factors with eigenvalues greater than 1, which accounted for 66.92 % of the total variance. The first factor included 9 items (Items 1-9), which explained 53.81% of the total variance (eigenvalue=5.38). The second factor included 3 items (Items 10-12), which explained 13.11% of the total variance (eigenvalue=1.31). Table 5 shows the component matrix for two factors extracted with the PCA method. Figure 1 depicts the scree plot for 12-item P-NNS.

 

Table 4. The kappa values for each item of the persian version of NNS

Items

Kappa values

Agreement power

Item 1

1

Excellent

Item 2

1

Excellent

Item 3

0.887

Excellent

Item 4

0.886

Excellent

Item 5

0.926

Excellent

Item 6

0.886

Excellent

Item 7

0.935

Excellent

Item 8

0.888

Excellent

Item 9

0.953

Excellent

Item 10

0.876

Excellent

Item 11

0.783

Good

Item 12

0.959

Excellent

 NNS: Non-Nutritive Sucking scoring system.

 

Table 5. The factor structure of the persian version of the NNS

Items

Factors

Factor 1

(positive items)

Factor 2

(negative items)

Item 1

0.662

 0.151

Item 2

0.637

 0.199

Item 3

0.804

-0.193

Item 4

0.852

-0.0098

Item 5

0.906

-0.102

Item 6

0.870

-0.054

Item 7

0.933

-0.086

Item 8

0.875

-0.227

Item 9

0.880

-0.120

Item 10

0.230

0.742

Item 11

0.420

0.570

Item 12

-0.193

0.709

NNS: non-nutritive sucking scoring system.

 

Discussion
The most important finding of the present study was that the NNS successfully translated and cross-culturally adapted to Persian language equivalent conceptually with the original instrument. This study showed that the P-NNS is a reliable and valid instrument for evaluating the oral feeding in preterm infants, with psychometric properties in agreement with the original English version.

Face validity and acceptability
The process of forward and back translation for the development of the P-NNS to ensure semantic and conceptual equivalence of the P-NNS to the original English version was proceeded without difficulties. An agreement among the translators and expert committee members was easily found. No major problems were identified in the pilot testing phase for evaluating the pre-final version reflected in responding easily to all items of the P-NNS. Feedback from SLPs about the content adequacy and wording clarity of P-NNS confirmed the face validity and acceptability of the P-NNS. The face validity and acceptability of P-NNS were consistent with those reported for original English (17) and Chinese (20).

Floor and ceiling effects
Distribution of scores for P-NNS was well ranged without any floor or ceiling effect. Neither a ceiling nor floor effect was identified as no preterm infant had a maximal or minimal P-NNS score. No floor or ceiling effect identified in this study reveals the ability of P-NNS in the meaningful detection of deterioration or improvement in infant’s oral feeding. Excellent completeness of item response by patients, well score distribution, and absence of floor or ceiling effect support the content validity and responsiveness of the P-NNS. Floor and ceiling effects have not been reported for the original English (17) and Chinese (20) versions of the NNS.

Internal consistency reliability 
The P-NNS showed high acceptable internal con-sistency (α=0.905) similar to that (α=0.90) reported for the Chinese version (20) which confirms that all the items of P-NNS are interrelated as a homogenous instrument. The Cronbach’s α for P-NNS that was between the proposed criterion (0.70–0.90) for a good internal consistency reliability (29) indicates no redundancy in P-NNS items.

The SEM and SDC
The SEM and SDC are the absolute reliability measures providing information about the reliability of the instruments and enables clinicians to make well informed decisions regarding whether a real change after an intervention has occurred or whether the observed change is due to the measurement error.
The value of SEM for P-NNS was low that indicates a small magnitude of measurement error reflecting the obtained score is more likely to be closer to the true value. The low SEM observed in this study suggests the potential for high test–retest reliability of the P-NNS. In this study, we did not use a test–retest reliability. The SEM obtained for the P-NNS is low that verifies it as a reliable and sensitive measure for detecting real changes. The Chinese study with the NNS had used a test–retest design and confirmed the retest reliability of the Chinese version of the NNS (20).
The SDC enables clinicians to judge whether the differences of an individual patient achieved on the P-NNS are real. The SDC value of the P-NNS was 22.28. To determine whether a real change in outcome has occurred between testing sessions using the P-NNS, changes in scores achieved by a preterm infant with feeding problems should be more than or less than 22.28 to be considered real. The SEM and SDC were not calculated in the previous studies (17,20).

Inter-rater reliability
This study demonstrated that the P-NNS had ex-cellent inter-rater reliability for evaluating oral feeding readiness in preterm infants. Our findings are consistent with those found for the original English (17). Inter-rater validity was not investigated in the Chinese study (20).

Construct validity 
In the current study, construct validity was examined in terms of the correlation between the P-NNS and Persian version of the EFS. A significant inter-correlation between the P-NNS and Persian version of the EFS was revealed, which indicates that the P-NNS has an excellent convergent validity. Convergent construct validity was not calculated in other studies (17,20).

Factor analysis
A factor analysis performed to determine the latent components of P-NNS returned the two latent factors (positive items and negative items). The first factor under the title of positive items was composed of items 1 to 9. The second factor under the title of negative items consisted of items 10 to 12. These findings were in line with the results of original English and Chinese versions of the NNS studies (17,20).
This study has some limitations. The discriminative validity and cut-off score were not investigated for the P-NNS score. Moreover, we have not evaluated the test–retest reliability based on repeated measurements obtained by P-NNS. Future studies are needed to investigate the discriminative validity and cutoff score of the P-NNS in Persian’s preterm population and usefulness of P-NNS for repeated assessments of preterm infants over time. 

Conclusion
The present study translated the NNS into the Persian language following standard guidelines and provided evidence of reliability and validity. The Persian NNS is now available for use in NICU and outpatient settings to assess non-nutritive sucking and oral feeding readiness of preterm infants.

Acknowledgements
We acknowledge physicians, nurses, and parents of premature infants in the Akbarabadi Teaching Hos-pital. This study has been approved by the Research Ethics Committee at the Iran University of Medical Sciences (ethics code: ethics code: IR.IUMS.REC. 1399.1430).

Data availability
Data are available on request from the authors.

Conflict of Interest
All authors declare that they have no conflicts of interest.

  1. References

    1. Jenik AG, Vain N. The pacifier debate. Early Hum Dev 2009 Oct 1;85(10):S89-91. https://pubmed.ncbi.nlm.nih.gov/19762175/
    2. Medoff-Cooper B, Shults J, Kaplan J. Sucking behavior of preterm neonates as a predictor of developmental outcomes. J Dev Behav Pediatr 2009 Feb 1;30(1):16-22. https://pubmed.ncbi.nlm.nih.gov/19194323/
    3. Sameroff AJ. The components of sucking in the human newborn. J Exp Child Psychol 1968 Dec 1;6(4):607-23. https://pubmed.ncbi.nlm.nih.gov/5686410/
    4. Crowe L, Chang A, Wallace K. Instruments for assessing readiness to commence suck feeds in preterm infants: effects on time to establish full oral feeding and duration of hospitalisation. Cochrane Database Syst Rev 2012 Apr 18;(4):CD005586. https://pubmed.ncbi.nlm.nih.gov/22513933/
    5. Wolff PH. The serial organization of sucking in the young infant. Pediatrics 1968 Dec 1;42(6):943-56. https://pubmed.ncbi.nlm.nih.gov/4235770/
    6. Johnson S. Cognitive and behavioural outcomes following very preterm birth. Semin Fetal Neonatal Med 2007 Oct;12(5):363-73. https://pubmed.ncbi.nlm.nih.gov/17625996/
    7. Jadcherla SR, Shaker R. Esophageal and upper esophageal sphincter motor function in babies. Am J Med 2001 Dec 3;111(8):64-8.
    8. Muglia LJ, Katz M. The enigma of spontaneous preterm birth. N Engl J Med 2010 Feb 11;362(6):529-35. https://pubmed.ncbi.nlm.nih.gov/20147718/
    9. Comrie JD, Helm JM. Common feeding problems in the intensive care nursery: maturation, organization, evaluation, and management strategies. Semin Speech Lang 1997;18(3):239-60; quiz 261. https://pubmed.ncbi.nlm.nih.gov/9306518/
    10. Lau C, Hurst N. Oral feeding in infants. Curr Probl Pediatr 1999 Apr 1;29(4):105-24. https://pubmed.ncbi.nlm.nih.gov/10202630/
    11. Alidad A, Tarameshlu M, Ghelichi L, Haghani H. The effect of non-nutritive sucking combined with oral motor stimulation and oral support on feeding performance in premature infants: A single-blind randomized–clinical trial. J Pediatr Rehabil Med 2021 Jan 1;14(3):379-87. https://pubmed.ncbi.nlm.nih.gov/34511515/
    12. Arvedson JC. Swallowing and feeding in infants and young children. GI Motility online. 2006 May 16.
    13. Lau C. Development of suck and swallow mechanisms in infants. Ann Nutr Metab 2015 Jul 1;66(Suppl. 5):7-14. https://pubmed.ncbi.nlm.nih.gov/26226992/
    14. Pinelli J, Symington AJ. Non-nutritive sucking for promoting physiologic stability and nutrition in preterm infants. Cochrane Database Syst Rev 2005 Oct 19;(4):CD001071. https://pubmed.ncbi.nlm.nih.gov/16235279/
    15. Harding C, Law J, Pring T. The use of non-nutritive sucking to promote functional sucking skills in premature infants: an exploratory trial. Infant 2006;2(6):238-43.
    16. Nijhuis JG. Fetal behavior. Neurobiol Aging 2003 May 1;24:S41-6. https://pubmed.ncbi.nlm.nih.gov/12829106/
    17. Neiva FC, Leone C, Leone CR. Non-nutritive sucking scoring system for preterm newborns. Acta Paediatr 2008 Oct;97(10):1370-5. https://pubmed.ncbi.nlm.nih.gov/18647278/
    18. Braun MA, Palmer MM. A pilot study of oral-motor dysfunction in “at-risk” infants. Physic Occup Ther Pediatr 1985 Jan 1;5(4):13-26.
    19. Neiva FC, Leone CR, Leone C, Siqueira LL, Uema KA, Evangelista D, et al. Non-nutritive sucking evaluation in preterm newborns and the start of oral feeding: a multicenter study. Clinics 2014;69:393-7.
    20. Wang Yu-mei ZL, Qiong LE, Li-fen WU, Xiong Li-juan, Yang Xue-jiao, Zhang Xue-hui, Reliability and Validity of Chinese Version of Non-nutritive Sucking Scoring System in Preterm Newborns. J Nurs 2019;26(3):62-65.
    21. Bahrami B, Marofi M, Farajzadegan Z, Barekatain B. Validation of the early feeding skills assessment scale for the evaluation of oral feeding in premature infants. Iran J Neonatol 2019;10(2):68-75.
    22. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine 2000 Dec 15;25(24):3186-91. https://pubmed.ncbi.nlm.nih.gov/11124735/
    23. Kristjansson, E.A., A. Desrochers, and B. Zumbo, Designer’s corner-translating and adapting measurement instruments for cross-linguistic and cross-cultural research: a guide for practitioners. Can J Nurs Res 2003 Jun;35(2):127-42. https://pubmed.ncbi.nlm.nih.gov/12908201/
    24. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007 Jan 1;60(1):34-42. https://pubmed.ncbi.nlm.nih.gov/17161752/
    25. Streiner DL. Starting at the beginning: an introduction to coefficient alpha and internal consistency. J Pers Assess 2003 Feb 1;80(1):99-103. https://pubmed.ncbi.nlm.nih.gov/12584072/
    26. Cicchetti DV, Sparrow SA. Developing criteria for establishing interrater reliability of specific items: applications to assessment of adaptive behavior. Am J Ment Defic 1981 Sep;86(2):127-37. https://pubmed.ncbi.nlm.nih.gov/7315877/
    27. Fleiss, J.L., B. Levin, and M.C. Paik, Statistical methods for rates and proportions. 2013: john wiley & sons.
    28. Costello AB, Osborne J. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract Assess Res Evaluation 2005;10(1):7.
    29. Cortina, J.M., What is coefficient alpha? An examination of theory and applications. J Appl Psychol 1993 Feb;78(1):98.