Skip to main content
Critical Appraisal Worksheets [click files to download - IM Journal Club uses blue McMaster sheets]
Types of Follow Up
Intention to Treat - Once Randomized, Always Analyzed; participants included even if they didn’t fully adhere to protocol
Per Protocol – only patients who complete the entire clinical trial according to protocol are counted towards final results. Can lead to attrition bias if dropout was uneven. Tends to give an "ideal scenario" of how treatment works
Blinding & Allocation
Blinding – two or more parties in a trial are kept unaware of which treatment arms the participants have been assigned to
- Can be blinded at different levels (unblended; single blinded with patient; or double blinded with clinician and patient unaware of treatment)
Allocation – person randomizing patient doesn’t know what the next treatment allocation will be
Types of Error
Type 1 Error: False Positive Rate (detects an effect that isn’t present)
Type 2 Error: False Negative Result (failure to detect an effect that is present)
Example: The Boy who cried wolf
Common Types of Bias
- Selection Bias – select sample/data wrong, usually because data is easy to access; accidentally work with specific subset of patients rather than the whole (sample is unrepresentative of the whole population)
- Self-selection Bias – found in polling, surveys; Subcategory of selection bias; Let the subjects of analyses select themselves – less proactive people will be excluded
- Recall Bias – common in interviews/surveys. Respondent doesn’t remember things correctly (or can’t); not about having a good or bad memory – humans have selective memory by default
- Observer Bias - Researcher subconsciously projects their own expectations onto the research
- Funding Bias (sponsorship bias) - Scientific studies are biased in a way that supports the financial sponsor of the research
- Survivorship Bias - Researcher focuses only on the part of the data set that already went through some kind of pre-selectin process – and missing the data points that fell of during this process because they’re not visible anymore
- Cognitive Bias
- Hindsight Bias – even the greatest findings seem very trial looking back on them a few days later
- Confirmation Bias – carrying pre-conceptions in to research & looking only to confirm it
- Belief Bias – someone is so sure of gut feelings that they ignore results of a data research project
- Curse of knowledge – assuming someone else has the same background knowledge you
- Omitted Variable Bias - Occurs when you are leaving out one or more important variables from your model – especially often regarding predictive analytics
- Cause-Effect Bias – correlation does not imply causation
- Publication Bias - When the likelihood of a study being published is affected by the findings of the study.
- Language bias - English language has been the predominant language in medical research. Studies publishing positive results might also be more likely to publish in English. Reading and using only English language research could provide a biased assessment of a topic, and can lead to biased results
- Confounding Variable Bias – has direction (positive – biased away from null, overestimates effect; or negative – biased toward null, underestimates effect)
- Confounding Variable - Extra variables in a study that weren’t accounted for
Propensity score matching – Drawing causal conclusions from observational studies where the “treatment” was not randomly assigned
- Approximates a random trial to match controls with experimental subjects; control group is given a propensity score and matched with at least 1 participant.
- Bipartate matching – equivalent to sampling without a replacement
- Non-bipartate matching – sampling with replacement (reuse a member, such as using control as a match for two or more treatment group participants)
- Can be used to control confounding; other methods include stratification, regression adjustment, and weighting.
(more info: https://catalogofbias.org/biases/)
Types of Questions & study design
DETERMINING THE BEST STUDY TO SEARCH FOR DURING YOUR LITERATURE SEARCH.
|Type of Question
||Description (What is it?)
||Study Design (when SR not available)
An evaluation of a test, screening or other assessment such as history or physical exam
Prospective, blind comparison to a gold standard.
RCT > controlled clinical trial (CCT) > cohort > case control > case series
An evaluation of a therapeutic or preventive intervention
RCT > CCT > cohort >case control > case series
||An evaluation of clinical outcomes over time
||cohort > case control > case series
An evaluation of a therapeutic, preventive, screening or diagnostic intervention, or a non-therapeutic exposure or behavior
cohort > case control > case series
An evaluation of the effectiveness of an intervention or exposure in preventing morbidity and mortality
|RCT > CCT > cohort > case control > case series
An evaluation of cost versus benefit of a treatment or procedures
|Economic analysis, cost-benefit analysis
Contact your Clinical Librarian
Contact the Darnall Medical Library (DML)
Users' Guides to the Medical Literature
General Critical Appraisal Resources
Helps decision makers identify the best available evidence by providing guides to the systematic consideration of the validity, importance, and applicability of claims about the assessment of health problems and the outcomes of health care.
Statistics for the Non-Statistician
Basic and Clinical Biostatistics by A comprehensive user-friendly introduction to biostatistics and epidemiology applied to medicine, clinical practice, and research.
Publication Date: 2004-04-15
Statistics in Medicine by Statistics in Medicine, Third Edition makes medical statistics easy to understand by students, practicing physicians, and researchers. The book begins with databases from clinical medicine and uses such data to give multiple worked-out illustrations of every method. The text opens with how to plan studies from conception to publication and what to do with your data, and follows with step-by-step instructions for biostatistical methods from the simplest levels (averages, bar charts) progressively to the more sophisticated methods now being seen in medical articles (multiple regression, noninferiority testing).
Call Number: WA 950 R564s 2012
Publication Date: 2012-07-09
Darnall Medical Library | Walter Reed NMMC | Building 1, Room 3458 | 8955 Wood Road | Bethesda, MD 20889 | 301-295-1184/85 | Open Monday-Friday, 0700-1730
After-hours access to the library is available to WRNMMC Staff via the CDO at 301-295-4611.