Psychiatry Portal: Calculating Results

Absolute Risk 

The likelihood of an event happening under specific conditions (usually a “1/10 chance” or “10% chance”) 

• # of events (good or bad) in treated or control groups, divided by the number of people in that group

Relative Risk (RR) 

The likelihood of an event occurring in a group of people compared to another group with different behaviors, physical conditions, or environments (usually a % increase/decrease compared to groups)

• RR = ART/ARC (# events in treatment/# events in control)

Relative Risk ratio 

The ratio of the probability of an event occurring in the exposed group versus the probability of the event occurring in the non-exposed group

Absolute Risk Reduction (ARR) – “risk difference” 

Shows exact difference from baseline, how much lower the modified risk is from the start risk in absolute terms

• ARR = ARC – ART (# events in control - # events in treatment)

Relative Risk Reduction (RRR) – relative number 

Tells us how much lower the modified risk is than the starting risk as a proportion. Can look very impressive for small differences.

• Calculated RRR = 1-RR  OR (ARC – ART)/ARC

Hazard Ratio

HR is a comparison between the probability of events in a treatment group compared to the probability of events in a control group

• HR >1 means treatment group healed faster or had a slower time to event
• HR = 1 means that both groups are experiencing equal number of events at any point in time
  • Hazard ratio of 3 - 3x the number of events are seen in treatment group at any point in time; the treatment will cause the patient to progress three times as fast as patients in the control group
  • Hazard ratio of 0.333 – HR in treatment group is 1/3 of that in the control group
• Similar (but not the same as) relative risk ratio; RRR is cumulative over the entire period of the study; HR is at any particular point in time

Absolute Risk Reduction, Relative Risk Reduction, Relative Risk resources

• How to calculate a relative risk. Terry Shaneyfelt video (3 min)
• Basic Statistics for Clinicians: Assessing the effects of treatment: measures of association CMAJ. 1995 Feb 1;152(3):351-7. Erratum in: Can Med Assoc J 1995 Mar 15;152(6):813.

Odds Ratio

Compares odds of an exposure in cases to odds of exposure in controls

• Is also a measure of outcome vs exposure 

• Reasonable approximation of relative risk when the disease/outcome being studied is relatively rare (<10%) in people who have  not been exposed. 
• When used correctly, they have a similar meaning as relative risk
  • OR > 1, increased likelihood of the outcome
  • OR < 1 , decreased likelihood of the outcome
•  Odds Ratios can be calculated for case control studies (in which the prevalence of disease is for exposed vs unexposed population is not known)

Odds Ratios resources

• Gordon Guyatt explaining odds ratios video (21 min)
• How to calculate an odds ratio Terry Shaneyfelt video (3 min)
• Odds ratios Philip Sedgwick. BMJ 2010;341:c4414

Number needed to Treat (NNT)

The number of patients you need to treat to prevent one additional bad outcome (death, stroke, heart attack, etc.)

• NNT = 1/ARR

  • ARR = CER – EER (control event rate – experimental event rate)
• NNT are always rounded up to nearest whole number and accompanied as standard by 95% CI
  • Ex – if a drug reduces risk of a bad outcome from 50%

Sensitivity and Specificity

• Describe the performance of test in a population
• The sensitivity and specificity of a test are independent of the prevalence of the disease in the population for which it is being used.

Positive predictive value, Negative predictive value

PPV = # of true positives / (# of true positives + # false positives)  (top of 2x2 table)

NPV = true negative rate / (true negative rate + false negative rate)*100  (bottom of 2x2 table)

• The probability that the disease is truly present- PPV - or absent- NPV, given a positive (PPV) or negative (NPV) test result
• The positive and negative predictive values  for a test VARY according to the prevalence of disease in a population
  • Baye’s theorem

Sensitivity and PPV

• Sensitivity - What % of people with disease does test correctly identify?Positive In Disease
• PPV - If the test result is positive, what is the probability of disease?

Specificity and NPV

• Specificity - What % of people without disease does test ID correctly? Negative In Health
• NPV - If the test result is negative, what is the probability of NO disease?

Sensitivity, Specificity and Likelihood ratios resources

• Sensitivity & specificity - H. Gilbert Welch video (8 min)
• How to calculate sensitivity - Terry Shaneyfelt video (4 min)
• How to calculate specificity - Terry Shaneyfelt video (3 min)

Assess the potential utility of a particular diagnostic test & looks at how likely it is that a patient has a disease or condition

• LRs are basically a ratio of the probability that a test result is correct to the probability that the test result is incorrect.
  • LR+ =  sensitivity / 1- specificity  = TP/FP
  • LR-  = 1- sensitivity / specificity =  FN/TN

Understanding the numbers:

• LR >1 means that the test result is associated with the PRESENCE of disease.  
• LR <1 (between 0 and 1)  means that the test result is associated with the ABSENCE of disease.
• LR = 1 means that the test result does not affect the likelihood of disease (i.e.worthless clinically!)
  • LRs are ratios derived from sensitivity and specificity (which are independent of disease prevalence). 
  • Multiply pre test odds by the likelihood ratio to get post test odds
  • Convert post test odds to post test probability

Characteristics of LR

• They are not dependent on the prevalence of the target disease in a population, because they are calculated from sensitivity and specificity
• They allow the clinician to calculate the probability of disease, via calculating pre and post test ODDS, for a given test result. 
  • PPV and NPV do also, but they are prevalence dependent

LR resources

• What are likelihood ratios and how are they used Terry Shaneyfelt video (10 min)

Forest Plots

• How to interpret a forest plot. Terry Shaneyfelt video (5 min)
• Interpreting and understanding meta-analysis graphs--a practical guide. Aust Fam Physician. 2006 Aug;35(8):635-8. PubMed PMID: 16894442

​

Types of Heterogeneity:

• Clinical: Differences in participants, interventions or outcomes
• Methodological: Differences in study design, risk of bias
• Statistical: Variation in intervention effects or results

Measuring Heterogeneity

• Look at it with the eyeball test on a forest plot (you can draw a line through all the points)
• OR use χ² test or I²
  • χ² assumes null hypothesis that all studies are homogenous – gives us a p-value to test. If p-value is low, we can reject the hypothesis, and heterogeneity is present

I²

measures extent of heterogeneity. Low is good, high is bad for numbers. >50% is a bad combination of tests.

• 0% to 40%: might not be important
• 30% to 60%: moderate heterogeneity
• 50% to 90%: substantial heterogeneity
• 75% to 100%: considerable heterogeneity

Resources:

What is heterogeneity and is it important? BMJ 2007; 334 :94. Fletcher, J.

Deeks JJ, Higgins JPT, Altman DG (editors). Chapter 9: Analysing data and undertaking meta-analyses. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.

What is Heterogeneity and why does it matter? Students 4 Best Evidence

• Mon-Fri, 0900-1500, try our Live Chat Service
• Need help? Ask the Library
• Urgent Patient Care Query: Contact the Library
• Looking for a Book or Article? Contact the Library Technicians
• Literature Search / Reference Question? Contact the Librarians
• Conducting Research? Contact the Research Librarian
• Research Design & Publication: Contact Research Discovery
• Darnall Medical Library, Circulation Desk: 295-1184

• TRICARE Appointment Line: 855-227-6331
• Walter Reed NMMC Main Phone: 301-295-4611
• TRICARE Health Services (includes phone numbers)

Researching a Medical Condition?
Click the button below!

• MHS Medical Library (MML)
• USUHS Learning Resource Center
• Alexander T. Augusta Military Medical Center Library

Trip (Turning Research Into Practice) is a database that only searches sites that have evidence-based content.

Helps decision makers identify the best available evidence by providing guides to the systematic consideration of the validity, importance, and applicability of claims about the assessment of health problems and the outcomes of health care. Includes the Users' Guide to the Medical Literature and the Rational Clinical Examination, among other resources.

The Cochrane Library is a collection of six databases containing different types of high-quality, independent evidence to inform healthcare decision-making including:

P-Value

Probability that the null hypotheses gives for a specific experimental result to happen; goal of experiments is to disprove the null hypothesis

• Low (<0.05) – null hypothesis is unlikely, and has statistical significance
• Low p-value means a higher chance of hypothesis being true
• Post-hoc tests – probabilities in post-hoc tests are CUMULATIVE – so probability of each test must equal α / the number of comparisons to preserve the overall significance level. You can’t just combine them and look for a p =0.05; should also adjust confidence intervals if that’s the only piece given

95% confidence interval is a range of values that you can be 95% certain contains the true mean of the population.

Confidence Intervals

• Confidence intervals Terry Shaneyfelt video (5 min)

• Basic Statistics for Clinicians: Interpreting study results: confidence intervals CMAJ. 1995 Jan 15;152(2):169-73.

• more videos by Terry Shaneyfelt, M.D.
• more videos by Allan Shaughnessy
• Stating the meaning of effect size measures in plain English
• Endgames statistical series in BMJ by Philip Sedgwic
• Students4BestEvidence