You are viewing as a guest Login
 
     
  amstarbanner  
     
Home About Us Publications Checklist FAQs Contact Us    
 

Tool Name

Tool Description

Link to article/ tool PDF

Validation Process

The TREND  (Transparent Reporting of Evaluations with Nonrandomized Designs) Statement

A 22-item checklist developed to guide standardized reporting of non randomized controlled trials. It complements the CONSORT statement  which was developed for RCTs.

Des Jarlais, D. C., Lyles, C., Crepaz, N., &TRENDGroup. (2004). Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: TheTRENDstatement.  American Journal of Public Health, 94, 361-366.
http://www.cdc.gov/trendstatement/

Has information on Information on validated instruments such as psychometric and biometric properties but does not state if the TREND statement is validated itself

Newcastle-Ottawa Scale

Developed to assess the quality of nonrandomised studies with its design, content and ease of use directed to the task of incorporating the quality assessments in the interpretation of meta-analytic results. It allocates a maximum of nine stars, for quality of selection, comparability, exposure and outcome of study participants.

Wells G, Shea B, O’Connell D, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomized studies in meta-analysis.
http://www.ohri.ca/programs/clinical_epidemiology/nosgen.pdf

The authors of NOS state that the validity assessment of the scale is under development.

Quality Assessment Tool For Quantitative Studies
The Effective Public Health Practice Project

This tool was developed by The Effective Public Health Practice Project for use in public health, and can be applied to articles of any public health topic area. It consists of seven steps.

Effective Public Health Practice Project Quality Assessment Tool 2003.
http://www.ahs.uwaterloo.ca/~manske/Presentations/CTCRI%20BPworkshop/PHRED%20Criteria%20Tool%202003.pdf

The validation process involved assessing the instrument's content and categories for clarity, completeness and relevance, and an overall comparison with similar types of tools.

Checklist for Measuring Quality- Downs and Black

Provides both an overall score for study quality and a numeric score out of a possible 30 points. It has five sections.  Administration of the tool can happen either within a systematic review process, or as a quality assessment tool for individual articles.

Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 1998; 52:377–84.
http://jech.bmj.com/content/52/6/377.full.pdf+html

The checklist was revised and tested by comparing the Quality Index (total score) with the total score obtained using an existing validated checklist (Standards of Reporting Trials Group. A proposal for structured reporting of randomised controlled trails. JAMA 1994; 272:1926–31.)

GATE (Graphic Appraisal Tool for Epidemiology)

A visual framework that illustrates the generic design of all epidemiologic studies.

Jackson R, Ameratunga S, Broad J, Connor J, Lethaby A, Robb G, et al. The GATE frame: critical appraisal with pictures. Evid Based Med 2006;11: 35-8.
http://www.fmhs.auckland.ac.nz/soph/depts/epi/epiq/_docs/gateframe.pdf (pdf)

Not Validated

CriSTal Checklist

Evaluates the quality of various research designs, including appraising a user study or appraising information needs analysis.

Checklist for Appraising an Information Needs analysis :
http://nettingtheevidence.pbwiki.com/f/needs.doc
Checklist for Appraising a User Study :
http://nettingtheevidence.pbwiki.com/f/use.doc

Not stated

ReLIANT ( Readers guide to the Literature on Interventions Addressing the Need for education and training)

Aimed at appraising published reports of educational and training interventions.

Koufogiannakis, D., Booth, A. & Brettle, A. ReLIANT: Readers’s guide to the Literature on Interventions Addressing the Need for education and Training. Library and Information Research 2006, 30, 44–51.
http://eprints.rclis.org/7163/1/RELIANT__final_.pdf (pdf)

Not stated

EBLIP Critical Appraisal Checklist

The EBLIP checklist for library research provides a thorough, generic list of questions that one would ask when attempting to determine the validity, applicability and appropriateness of a study.

Booth, A. Eleven steps to EBLIP service. Health information and libraries journal, 2009, vol. 26, n° 1, p. 81-84.
http://www.nihs.ie/pdf/EBL Critical Appraisal Checklist.pdf

Not stated

READER (Relevance, Education, Applicability, Discrimination, Evaluation and Reaction)  Critical Appraisal Tool

Represents a sequence of steps in the assessment of general practice literature. It  involves the evaluation of the article using a scoring system where at the end of the article, the reader decides what to do with it

Domhall M, READER: an acronym to aid critical reading by general practitioners. British Journal of General Practice, 1994, 44, 83-85.
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1238789/pdf/brjgenprac00035-0039.pdf

Not stated

The STARD ( STAndards for the Reporting of Diagnostic accuracy ) Checklist

Consists of a 25 item checklist and flow diagram that helps to determine the accuracy and completeness of reporting of studies of diagnostic accuracy.

Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. The STARD Statement for reporting studies of diagnostic accuracy: explanation and elaboration. Clin Chem 2003; 49:7–18.
http://www.clinchem.org/content/49/1/7.full.pdf+html

Not stated

MINORS ( Methodological Index for Non Randomized Studies) Tool

A valid instrument consisting of 12 items designed to assess the methodological quality of non-randomized surgical studies. The first eight items are specifically for non-comparative studies.

Slim K, Nini E, Forestier D, Kwiatkowski F, Panis Y, Chipponi J. Methodological index for non-randomized studies (minors): development and validation of a new instrument. Aust NZ J Surg. 2003; 73:712–716.
http://onlinelibrary.wiley.com/doi/10.1046/j.1445-2197.2003.02748.x/pdf

Followed the principles of scale construction outlined by: Bland JM, Altman DG. Validating scales and indexes. BMJ 2002; 324: 606–7.

JBI-NOTARI ( Narrative, Opinion and Text Assessment and Review Instrument) Critical Appraisal Tool

Evaluates narrative, opinion and others textual evidence. However to use this tool a login is required.

To use this tool registration with JBI is requires
http://www.joannabriggs.edu.au/Appraise Evidence/JBI SUMARI (systematic review software)

Not stated

Modified set of Russell and Gregory’s criteria for methodological soundness

A nine item checklist separated into three themes (Finding validity, description and application)

Russell C, Gregory D. Evaluation of qualitative research studies. Evid Based Nurs. 2003; 6:36-40.
http://ebn.bmj.com/content/6/2/36.full

Not stated

Quality of Reporting of Observational Longitudinal Research

Comprises of a 33 item criteria and a flow diagram adapted from CONSORT. The criteria represent two principal categories: 1) aspects that could possibly influence effect estimates and 2) more descriptive or contextual elements

Tooth L, Ware R, Bain C, Purdie DM, and Dobson A. Quality of reporting of observational longitudinal research. Am J Epidemiol. 2005; 161(3):280-288.
http://aje.oxfordjournals.org/content/161/3/280.abstract

Two raters independently tested the final checklist on a random selection of articles
describing observational longitudinal research

GRADE system

Classifies the quality of evidence in one of four levels-high, moderate, low, and very low.  The GRADE system offers two grades of recommenda­tions: “strong” and “weak”

Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008; 336:924-6.
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2335261/?tool=pubmed

Not stated

Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center program

Panel of 5-8 experts from 13 chosen centers convene and develop a topic's key questions, provide advice on which types of studies to include or exclude, and suggest other analyses that may be useful.

Atkins D, Fink K, Slutsky J. Better information for better health care: the Evidence-based Practice Center program and the Agency for Healthcare Research and Quality. Ann Intern Med 2005 Jun 21; 142(12 Pt 2):1035-1041.
http://www.annals.org/content/142/12_Part_2/1035.full.pdf

The Agency elected to emphasize broad, general expertise among the EPCs rather than establishing centers specializing in a single content area, such as cardiology. This EPC is now examining whether scores calculated by using this instrument are associated with reported adverse event
rates in other surgical and nonsurgical case series

Pengel scale

Specific to prospective studies. Six criteria are used to assess methodological quality.

Pengel  LHM, Herbert RD, Maher CG, Refshauge KM. Acute low back pain: systematic review of its prognosis. BMJ. 2003; 327(7410):323.
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC169642/pdf/el-gp323.pdf

Not stated

Modified methodological quality assessment tool which was developed and based on existing assessment tools

Consists of 2 checklists, one for studies of incidence or prevalence and another for risk factors. It has 6 items each that assess external validity. For internal validity, the checklist of incidence has 5 items and the risk factor checklist has 13 items

Shamliyan T.  Kane RL, Ansari MT, Raman G, Berkman ND, Grant M, Janes G, Magilione M, Moher D, Nasser M, Robinson KA, Segal JB, Tsouros S. Criteria to assess quality of observational studies evaluating the incidence, prevalence and risk factors of chronic diseases. In. 2009.
http://www.sciencedirect.com/science/article/pii/S089543561000301X

Conducted a pilot test of the checklists. The experts each evaluated 10 articles to test reliability and discriminant validity.  Examined discriminant validity by testing the hypothesis that our checklists can discriminate quality across studies and discriminate reporting vs. methodological quality

Loney criteria for critical appraisal of research articles on prevalence of disease

Used by health professionals to critically appraise research articles that estimate the prevalence or incidence of a disease or health problem.

Loney PL, Chambers LW, Bennett KJ, Roberts JG, Stratford PW. Critical appraisal of the health research literature: prevalence or incidence of a health problem. Chronic Dis Can 1998; 19(4): 170–176.

Not stated

Quality Assessment for Diagnostic Accuracy Studies (QUADAS)

Assesses the quality of diagnostic accuracy studies included in systematic reviews

Whiting P, Rutjes AW, Reitsma JB, et al. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol 2003; 3:25.
http://www.biomedcentral.com/content/pdf/1471-2288-3-25.pdf

Validation process includes piloting the tool on a small sample of published studies, focusing on the assessment of the consistency and reliability of the tool. It is also piloted in a number of diagnostic reviews. Regression analyses are used to investigate associations between study characteristics and estimates of diagnostic accuracy in primary studies.

QUADAS-2

QUADAS-2 is based on user feedback from the initial tool developed in 2003. It is made up of four domains namely and applied in 4 phases

Whiting P, Rutjes AWS, Westwood ME et al (2011)
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 155:529–536
http://www.annals.org/content/155/8/529.full.pdf+html

Not stated

STROBE Checklist (Strengthening the reporting of observational studies in epidemiology)

Consists of a checklist of 22 items, which relate to the title, abstract, introduction, methods, results, and discussion sections of articles. Eighteen items are common to cohort studies, case–control studies, and cross-sectional studies, and 4 are specific to each of the 3 study designs.

Von Elm, E. et al. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet 370, 1453–1457 (2007).
http://www.emgo.nl/kc/Analysis/statements/observationalstudies-lancet-2007.pdf

The generalisability (external validity) of the study results was discussed

RoBANS (Risk of Bias Assessment tool for Non-randomized Studies)

 

Contains 6 domains with scores of "low", "high" or "unclear".  It is harmonized with the Cochrane’s RoB tool and GRADE, and can be incorporated into RevMan and GRADEpro.

http://2011.colloquium.cochrane.org/abstracts/b8o3-risk-bias-assessment-tool-non-randomized-studies-robans-development-and-validation-ne

States it is Validated

Cochrane Risk of Bias tool for non-randomized studies

An extension of the current Cohrane risk of bias tool.

http://hiv.cochrane.org/sites/hiv.cochrane.org/files/uploads/Ch13_NRS.pdf

Not stated

A Scoring system for Mixed Methods Research and Mixed Studies Reviews

A new form of literature review has emerged, Mixed Studies Review. These Assesses Qualitative (6 items), Quantitative experimental (3 items), Quantitative observational (3 items) and Mixed Methods (3 items).

Pluye P, Gagnon MP, Griffiths F, Johnson-Lafleur J. A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in Mixed Studies Reviews. IInternational Journal of Nursing Studies. 2009 Apr; 46 (4):529-46. http://www.ncbi.nlm.nih.gov/pubmed/19233357

Not Validated

The iCAHE Guideline Checklist

Consists of 14 questions and can be used either as a checklist or a total score. It provides clinicians with a quick way of appraising the quality of a clinical guideline.

http:///www.unisa.edu.au/cahe/Resources/iCAHEGuidelineCH/iCAHE_guideline_checklist.pdf

Not stated

The Appraisal of Guidelines for Research & Evaluation (AGREE II ) Instrument

Consists of 23 items. It assesses the quality of the guidelines, provides a methodological strategy for the development of guidelines; and aims to inform what information and how information ought to be reported in guidelines.

AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003 Feb; 12(1):18-23.
http://www.agreetrust.org/index.aspx?o=1397

Face, construct and criterion validity were measured.  Attitudes about the instrument and user guide were collected by questionnaire. Assessments of criterion validity were assessed by calculating the Kendall’s tau B rank correlation coefficients between the appraisers’ domains scores and the overall assessment scores

Critical Appraisal Skills Programme (CASP)

CASP: Cohort Studies

Critical Appraisal Skills Program (CASP): Cohort Studies is a methodological checklist which provides key criteria relevant to cohort studies

CASP, NHS. Critical Appraisal Skills Programme (CASP): appraisal tools. Public Health Resource Unit, NHS 2003.
http://www.sph.nhs.uk/sph-files/cohort12questions.pdf

Not stated

CASP: Economic Evaluation Studies

Critical Appraisal Skills Program (CASP): Economic Evaluation Studies is a methodological checklist which provides key criteria relevant to economic studies.

Drummond MF, Stoddart+ GL, Torrance GW. Methods for the economic evaluation of health care programmes. Oxford: Oxford University Press, 1987.
http://www.sph.nhs.uk/sph-files/Economic Evaluations 10 Questions.pdf

Not stated

CASP: Diagnostic Test Studies

Critical Appraisal Skills Program (CASP): Diagnostic Test Studies is a methodological checklist which provides key criteria relevant to diagnostic studies.

Jaesche R, Guyatt GH, Sackett DL, Users’ guides to the medical literature, VI. How to use an article about a diagnostic test. JAMA 1994; 271 (5): 389-391
http://www.sph.nhs.uk/sph-files/Diagnostic Tests 12 Questions.pdf

Not stated

CASP: Case Control Studies

Critical Appraisal Skills Program (CASP): Case Control Studies is a methodological checklist which provides key criteria relevant to case control studies.

http://www.sph.nhs.uk/sph-files/Case Control 11 Questions.pdf

Not stated

CASP: Qualitative Research

Critical Appraisal Skills Program (CASP): Qualitative Research is a methodological checklist which provides key criteria relevant to qualitative research studies.

http://www.sph.nhs.uk/sph-files/Qualitative Appraisal Tool.pdf (pdf)

Not stated

CASP: Systematic Reviews

Critical Appraisal Skills Program (CASP): Systematic Reviews is a methodological checklist which provides key criteria relevant to systematic reviews.

Oxman AD, Cook DJ, Guyatt GH, Users’ guides to the medical literature. VI. How to use an overview. JAMA 1994; 272 (17): 1367-1371
http://www.sph.nhs.uk/sphfiles/S.Reviews%20Appraisal%20Tool.pdf

Not stated

Therapy Critical Appraisal Worksheet

Therapy CA Worksheet

Therapy Critical Appraisal Worksheet is a methodological checklist which provides key criteria relevant to therapy studies.

Straus SE, Richardson WS, Glasziou P, Haynes RB. Evidence based medicine. How to practice and teach it. Fourth Edition. Churchill Livingstone: Edinburgh, 2010. 0-702-03127-5, 312 pages
http://ktclearinghouse.ca/cebm/teaching/worksheets/therapy

Not stated

Diagnostic CA Worksheet

Diagnostic Critical Appraisal Worksheet is a methodological checklist which provides key criteria relevant to diagnostic studies.

Straus SE, Richardson WS, Glasziou P, Haynes RB. Evidence based medicine. How to practice and teach it. Fourth Edition. Churchill Livingstone: Edinburgh, 2010. 0-702-03127-5, 312 pages
http://ktclearinghouse.ca/cebm/teaching/worksheets/diagnosis

Not stated

Harm CA Worksheet

Harm Critical Appraisal Worksheet is a methodological checklist which provides key criteria relevant to harm studies.

Straus SE, Richardson WS, Glasziou P, Haynes RB. Evidence based medicine. How to practice and teach it. Fourth Edition. Churchill Livingstone: Edinburgh, 2010. 0-702-03127-5, 312 pages
http://ktclearinghouse.ca/cebm/teaching/worksheets/harm

Not stated

Prognosis CA Worksheet

Prognosis Critical Appraisal Worksheet is a methodological checklist which provides key criteria relevant to prognostic studies.

Straus SE, Richardson WS, Glasziou P, Haynes RB. Evidence based medicine. How to practice and teach it. Fourth Edition. Churchill Livingstone: Edinburgh, 2010. 0-702-03127-5, 312 pages
http://ktclearinghouse.ca/cebm/teaching/worksheets/prognosis

Not stated

Systematic Review (of Therapy) Worksheet

This methodological checklist provides key criteria relevant to systematic reviews.

Straus SE, Richardson WS, Glasziou P, Haynes RB. Evidence based medicine. How to practice and teach it. Fourth Edition. Churchill Livingstone: Edinburgh, 2010. 0-702-03127-5, 312 pages
http://ktclearinghouse.ca/cebm/teaching/worksheets/sr

Not stated

Scottish Intercollegiate Guidelines Network (SIGN)

SIGN Checklist 1: Systematic Reviews and Meta-analyses

Identifies the study, the reviewer, the guideline for which the paper is being considered as evidence, and the key question(s) it is expected to address, Relates to the overall assessment of the paper

Scottish Intercollegiate Guidelines NetworkISBN 978 1 905813 81 0First published December 2011
http://www.sign.ac.uk/guidelines/fulltext/50/checklist1.html

Validity is based on the themes of credibility and accountabilityi.e. the link between a set of guidelines and the scientific evidence must be explicit, and scientific and clinical evidence should take precedence over expert judgement. (Field 1990 http://www.nap.edu/openbook.php?record_id=1626&page=19 )

All SIGN guidelines are considered for review three years after publication.
(SIGN 50 handbook) http://www.sign.ac.uk/pdf/sign50.pdf

SIGN checklist 3: Cohort Studies

Designed to answer questions of the type “What are the effects of this exposure?”, It relates to studies that compare a group of people with a particular exposure with another group who either have not had the exposure, or have a different level of exposure.

Scottish Intercollegiate Guidelines NetworkISBN 978 1 905813 81 0First published December 2011
http://www.sign.ac.uk/guidelines/fulltext/50/checklist3.html

SIGN Checklist 4: Case-control Studies

Assesses studies that are generally used to assess the causes of a new problem, but may also be useful for the evaluation of population based interventions such as screening.

Scottish Intercollegiate Guidelines NetworkISBN 978 1 905813 81 0First published December 2011
http://www.sign.ac.uk/guidelines/fulltext/50/checklist4.html

SIGN Checklist 5: Diagnostic Studies

It has 3 sections. It identifies the study and makes a series of statements that are used to assess the internal validity of the study and rates the methodological quality of the study, based on the responses in the first section

Scottish Intercollegiate Guidelines NetworkISBN 978 1 905813 81 0First published December 2011
http://www.sign.ac.uk/guidelines/fulltext/50/checklist5.html

McMaster Critical Review Form

 Quantitative Studies

McMaster Critical Review Form for Qualitative studies contains a generic quantitative appraisal tool, accompanied by detailed guidelines for usage.

Law M, Stewart D, Pollock N, et al. Guidelines for Critical Review Form: Quantitative Studies. Hamilton, Ontario: McMaster University Publications, 1999:1–11.
http://www.srsmcmaster.ca/Portals/20/pdf/ebp/quanreview.pdf

Not stated

Qualitative Studies

McMaster Critical Review Form for Qualitative studies contains a generic quantitative appraisal tool, accompanied by detailed guidelines for usage.

Letts, L., Wilkins, S., Law, M., Stewart, D., Bosch,
J., & Westmorland, M. (2007). Critical Review
Form—Qualitative Studies (Version 2.0). Retrieved March 21, 2008
http://www.srsmcmaster.ca/Portals/20/pdf/ebp/qualreview_version2.0.pdf

Not stated

Evaluation tool

Evaluation Tool for Mixed Methods Studies

The evaluation tool for mixed studies allows appraisal of both the qualitative data collection and analysis component and the wider quantitative research design. It is applicable where the aim of the qualitative component is to draw out the informants' understandings and perceptions

Long AF, Godfrey M, Randall T, Brettle AJ and Grant MJ (2002) Developing Evidence Based Social Care Policy and Practice. Part 3: Feasibility of Undertaking Systematic Reviews in Social Care. Leeds: Nuffield Institute for Health
http://usir.salford.ac.uk/13070 (htm)

Not stated

Evaluation Tool for Quantitative Research Studies

The Evaluation Tool for Quantitative Studies contains six sub-sections: study evaluative overview; study, setting and sample; ethics; group comparability and outcome measurement; policy and practice implications; and other comments.

Long, A. & Godfrey, M. (2004), ‘‘An evaluation tool to assess the quality of qualitative research
Studies’’.  International Journal of Social Research Methodology, Vol. 7, 181_/196.
http://usir.salford.ac.uk/12969

Not stated

    Download Copy <PDF> <DOC>  
 
 
Copyright © 2021 AMSTAR All Rights Reserved