Reviewing the Reviewers: Critical Appraisal of the United States Veterans Health Administration Report on Femtosecond Laser Assisted–Cataract Surgery
Purpose
To assess the reporting and methodological quality of the United States Veterans Health Administration’s (VHA) 2013 systematic review on femtosecond laser-assisted cataract surgery (FLACS). As the VHA is the largest single provider of health care training in the US, this report will influence the adoption of FLACS by future ophthalmologists.
Methods
Two authors independently assessed the VHA Quality Enhancement Research Initiative’s (QUERI) 2013 systematic review, Benefits and Harms of Femtosecond Laser Assisted Cataract Surgery, using checklists derived from the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) and Assessment of Multiple Systematic Reviews (AMSTAR) guidelines for evaluating reporting characteristics and methodological quality, respectively. Scores were determined as a proportion of items fulfilled divided by total number of applicable items in each rubric. Two non-applicable items – Additional Analyses, Results and Additional Analyses, Methods – were removed from the PRISMA score denominator.
Results
The PRISMA and AMSTAR scores were 22 out of 25 (88%) and 9 out of 11 (82%), respectively. Deficiencies in reporting quality included non-registration of study protocol and lack of analyses for risk of bias across studies. Shortcomings in methodological quality included a lack of a list of excluded studies and absence of publication bias assessments.
Conclusion
While the reporting and methodology of the VHA QUERI systematic review were of good quality, future revisions should provide transparent research protocols and analyses of risk of bias. With the increasing prevalence of FLACS, rigorous systematic reviews will be critical to monitoring its outcomes in teaching hospitals and other settings.