A Note on the Importance of Quality Assessment in Systematic Review: Letter to the Editor

We read the following paper entitled “Telehealth Systems for Midwifery Care Management during the COVID-19 Pandemic: A Systematic Review” by Shamsabadi et al, in Journal of Iranian Medical Council, Volume 6, Issue 2, Spring 2023 with interest (1). We feel that while the manuscript addresses a number of important points regarding the provision of perinatal care during the COVID-19 era, there are several concerns considering methodology warranting the discussion. 
This manuscript focused on collecting data about midwifery care management during Covid-19, particularly related to telehealth applications for care delivery. Our research team has conducted several systematic reviews, and are they aware of the importance of proper assessment of quality of each of the manuscripts included in the review. Quality assessment can be somewhat complicated: each type of article has a different method for determination of quality, reliability, and generalizability of the manuscript (2,3). Thus, the present systematic review collects data from various types of articles including randomized clinical trials, qualitative studies, observational studies (including time series, cross-sectional study, retrospective cohort study, case report, and a pilot study, cross sectional and review). While inclusion of each of these manuscript subtypes can be acceptable, the substantial heterogeneity in article type further necessitates proper quality assessment of each of the included manuscripts. 
Proper quality assessment in systematic review warrants further discussion, and it is the subject of ongoing investigation. For example, Vo et al conducted an analysis of systematic reviews associated with mediation studies. These authors demonstrated that 23% of the reviews did not assess risk of bias at all, and 47% of the included studies used bias assessment tools that were not specifically designed for mediation studies. Perhaps, even more concerningly, 30.1% of the studies assessed the risk of bias using purely narrative means or with non-validated tools (4). Tran et al corroborate these results in the dental literature, demonstrating a high prevalence of utilization of author-developed tools to assess the quality of included manuscripts. They conclude by stating the need for the development of comprehensive guidelines for systematic reviews specifically within each field of research to more precisely assess the quality of the evidence included in the review (5). 
While currently there is no widely accepted form of guidelines used to curate quality assessment for systematic reviews, several quality assessment tools have been broadly validated and used across the literature. The Cochrane Risk of Bias 2 (RoB 2) tool is utilized to evaluate Randomized Controlled Trials (RCTs) included in systematic reviews (6). The tool consists of six domains, including selection bias, performance bias, detection bias, attrition bias, reporting bias, and other biases. The authors of the systematic review evaluate each study and list the potential for bias as being either high, low, or unclear. Another frequently used quality assessment tool for systematic review is assessing the methodological quality of systematic reviews (AMSTAR) instrument. In lieu of evaluation of each individual article, the original AMSTAR appraises the overall quality of a certain systematic review of RCTs. It has been further updated to remain applicable for non-randomized studies of various healthcare interventions in addition to RCTS. The updated version of the instrument, the AMSTAR 2, consists of 16 items and provides the addition of a comprehensive user guide which instructs the investigator in the proper use of the tool. The AMSTAR provides an overall score based on limitations of a certain systematic review in critical domains relevant to the interpretation of the study (7). While neither of the above assessment of bias tools may have been ideal for the study conducted by Shamsabadi et al, the authors should explain their evaluation protocols, ideally using a well-validated tool tailored appropriately to the specific article type in question.                 
In addition to careful description of the methods used for quality assessment, the authors should more completely define cutoff points and exclusion and inclusion criteria used for article selection. Moreover, the type of technology used to provide telehealth services should be described in as detailed a manner as possible. The related applications for telehealth care provision warrant further discussion regarding their respective capabilities and requirements. The authors of this study have not clarified the methods used for quality assessment and have poorly defined the reasoning behind selection of various article types.
As an additional erratum in the text, the references in table 1 include up to the number 40, while the reference list at the end of the paper only includes 32 references. Overall, we commend the authors on an interesting contribution. However, we feel that a formal published addendum including more complete descriptions of quality assessment and revisions to the reference numbering would greatly enhance the scientific value of the manuscript.