Skip to Main Content

Psychiatry Portal: Calculating Results

Risk

 

undefinedAbsolute Risk 

The likelihood of an event happening under specific conditions (usually a “1/10 chance” or “10% chance”) 

  • # of events (good or bad) in treated or control groups, divided by the number of people in that group

Relative Risk (RR) 

The likelihood of an event occurring in a group of people compared to another group with different behaviors, physical conditions, or environments (usually a % increase/decrease compared to groups)

  • RR = ART/ARC (# events in treatment/# events in control)

Relative Risk ratio 

The ratio of the probability of an event occurring in the exposed group versus the probability of the event occurring in the non-exposed group

Absolute Risk Reduction (ARR) – “risk difference” 

Shows exact difference from baseline, how much lower the modified risk is from the start risk in absolute terms

  • ARR = ARC – ART (# events in control - # events in treatment)

Relative Risk Reduction (RRR) – relative number 

Tells us how much lower the modified risk is than the starting risk as a proportion. Can look very impressive for small differences.

  • Calculated RRR = 1-RR  OR (ARC – ART)/ARC

Hazard Ratio

HR is a comparison between the probability of events in a treatment group compared to the probability of events in a control group

  • HR >1 means treatment group healed faster or had a slower time to event
  • HR = 1 means that both groups are experiencing equal number of events at any point in time
    • Hazard ratio of 3 - 3x the number of events are seen in treatment group at any point in time; the treatment will cause the patient to progress three times as fast as patients in the control group
    • Hazard ratio of 0.333 – HR in treatment group is 1/3 of that in the control group
  • Similar (but not the same as) relative risk ratio; RRR is cumulative over the entire period of the study; HR is at any particular point in time

Absolute Risk Reduction, Relative Risk Reduction, Relative Risk resources

Odds Ratio

Odds Ratio

Compares odds of an exposure in cases to odds of exposure in controls

  • Is also a measure of outcome vs exposure 

  • Reasonable approximation of relative risk when the disease/outcome being studied is relatively rare (<10%) in people who have  not been exposed. 
  • When used correctly, they have a similar meaning as relative risk
    • OR > 1, increased likelihood of the outcome
    • OR < 1 , decreased likelihood of the outcome
  •  Odds Ratios can be calculated for case control studies (in which the prevalence of disease is for exposed vs unexposed population is not known)

Odds Ratios resources

Number Needed to Treat

Number needed to Treat (NNT)

The number of patients you need to treat to prevent one additional bad outcome (death, stroke, heart attack, etc.)

  • NNT = 1/ARR

    • ARR = CER – EER (control event rate – experimental event rate)
  • NNT are always rounded up to nearest whole number and accompanied as standard by 95% CI
    • Ex – if a drug reduces risk of a bad outcome from 50%

Sn, Sp, PPV, NPV

Sensitivity and Specificity

  • Describe the performance of test in a population
  • The sensitivity and specificity of a test are independent of the prevalence of the disease in the population for which it is being used.

Positive predictive value, Negative predictive value

PPV = # of true positives / (# of true positives + # false positives)  (top of 2x2 table)

NPV = true negative rate / (true negative rate + false negative rate)*100  (bottom of 2x2 table)

  • The probability that the disease is truly present- PPV - or absent- NPV, given a positive (PPV) or negative (NPV) test result
  • The positive and negative predictive values  for a test VARY according to the prevalence of disease in a population
    • Baye’s theorem

Sensitivity and PPV

  • Sensitivity - What % of people with disease does test correctly identify?Positive In Disease
  • PPV - If the test result is positive, what is the probability of disease?

Specificity and NPV

  • Specificity - What % of people without disease does test ID correctly? Negative In Health
  • NPV - If the test result is negative, what is the probability of NO disease?

Sensitivity, Specificity and Likelihood ratios resources

  • Sensitivity & specificity - H. Gilbert Welch video (8 min)
  • How to calculate sensitivity - Terry Shaneyfelt video (4 min)
  • How to calculate specificity - Terry Shaneyfelt video (3 min)

Likelihood Ratio

Assess the potential utility of a particular diagnostic test & looks at how likely it is that a patient has a disease or condition

  • LRs are basically a ratio of the probability that a test result is correct to the probability that the test result is incorrect.
    • LR+ =  sensitivity / 1- specificity  = TP/FP
    • LR-  = 1- sensitivity / specificity =  FN/TN

Understanding the numbers:

  • LR >1 means that the test result is associated with the PRESENCE of disease.  
  • LR <1 (between 0 and 1)  means that the test result is associated with the ABSENCE of disease.
  • LR = 1 means that the test result does not affect the likelihood of disease (i.e.worthless clinically!)
    • LRs are ratios derived from sensitivity and specificity (which are independent of disease prevalence). 
    • Multiply pre test odds by the likelihood ratio to get post test odds
    • Convert post test odds to post test probability

Characteristics of LR

  • They are not dependent on the prevalence of the target disease in a population, because they are calculated from sensitivity and specificity
  • They allow the clinician to calculate the probability of disease, via calculating pre and post test ODDS, for a given test result. 
    • PPV and NPV do also, but they are prevalence dependent

LR resources

  • What are likelihood ratios and how are they used Terry Shaneyfelt video (10 min)

Forest Plot

Forest Plots

Heterogeneity

Types of Heterogeneity:

  • Clinical: Differences in participants, interventions or outcomes
  • Methodological: Differences in study design, risk of bias
  • Statistical: Variation in intervention effects or results

Measuring Heterogeneity

  • Look at it with the eyeball test on a forest plot (you can draw a line through all the points)
  • OR use χ² test or I²
    • χ² assumes null hypothesis that all studies are homogenous – gives us a p-value to test. If p-value is low, we can reject the hypothesis, and heterogeneity is present

measures extent of heterogeneity. Low is good, high is bad for numbers. >50% is a bad combination of tests.

  • 0% to 40%: might not be important
  • 30% to 60%: moderate heterogeneity
  • 50% to 90%: substantial heterogeneity
  • 75% to 100%: considerable heterogeneity

 

Resources:

What is heterogeneity and is it important? BMJ 2007; 334 :94. Fletcher, J.

Deeks JJ, Higgins JPT, Altman DG (editors). Chapter 9: Analysing data and undertaking meta-analyses. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.

What is Heterogeneity and why does it matter? Students 4 Best Evidence

 

 

Contact the Darnall Medical Library (DML)

For WRNNMC Healthcare
Providers and Staff
For WRNNMC Patients
  • TRICARE Appointment Line: 855-227-6331
  • Walter Reed NMMC Main Phone: 301-295-4611
  • TRICARE Health Services (includes phone numbers)

Researching a Medical Condition?
Click the button below!

Our Partners

Featured Resource

Trip (Turning Research Into Practice) is a database that only searches sites that have evidence-based content.


JAMAevidence

JAMAevidence

Helps decision makers identify the best available evidence by providing guides to the systematic consideration of the validity, importance, and applicability of claims about the assessment of health problems and the outcomes of health care. Includes the Users' Guide to the Medical Literature and the Rational Clinical Examination, among other resources.

ACP Journal Club

ACP Journal Club

The Cochrane Library

The Cochrane Library is a collection of six databases containing different types of high-quality, independent evidence to inform healthcare decision-making including:

P-Value & CI

P-Value

Probability that the null hypotheses gives for a specific experimental result to happen; goal of experiments is to disprove the null hypothesis

  • Low (<0.05) – null hypothesis is unlikely, and has statistical significance
  • Low p-value means a higher chance of hypothesis being true
  • Post-hoc tests – probabilities in post-hoc tests are CUMULATIVE – so probability of each test must equal α / the number of comparisons to preserve the overall significance level. You can’t just combine them and look for a p =0.05; should also adjust confidence intervals if that’s the only piece given

95% confidence interval is a range of values that you can be 95% certain contains the true mean of the population.


Confidence Intervals

Core EBM Books

 

Disclaimer: The appearance of hyperlinks does not constitute endorsement by the Department of Defense, Defense Health Agency of non-U.S. Government sites or the information, products, or services contained therein. Although the Darnall Medical Library, WRNMMC may or may not use these sites as additional distribution channels for Department of Defense information, it does not exercise editorial control over all of the information that you may find at these locations. Such hyperlinks are provided consistent with the stated purpose of this website.

Darnall Medical Library | Walter Reed NMMC | Building 1, Room 3458 | 8955 Wood Road | Bethesda, MD 20889 | 301-295-1184/85 | Open Monday-Friday, 0700-1630

After-hours access to the library is available to WRNMMC Staff via the CDO at 301-295-4611.