Internal Medicine Portal: Archived - Journal Club Blog

Block 2 RCT Wrap-Up - Biases (9/2020)

RCT Wrap-up (Block 2) - Bias 

In this block, we looked at critically appraising RCTs with a special focus on bias & confounding, power, and clinical vs statistical significance. Residents used a variety of well-known trials (ACCORD-BP, SPRINT, ANDROMEDA, and HYVET) to put these concepts to practice. Below is a summary of concepts related to bias and confounding. 

Bias is a systematic error. It doesn’t occur randomly. It is the result of doing something purposely different in one group in a study than is done in the other group(s) in a study. It can be looked at as the intentional or unintentional adjustment in design in a manner that may affect study results independent of the treatment effect. It creates an association that is not true. It could be in the conduct of a clinical trial or in the analysis or reporting of clinical data. 

Confounding is a very specific type of bias. It describes an association that is true but potentially misleading. A confounder affects both the dependent and independent variables, causing a spurious association. The association is spurious because it is not a causal association. For example, age or gender could affect results in aways that the trial is not set up to look for. 

Remember if you do detect a bias, you should determine whether the consequences of the bias are sufficiently large that they change the conclusions of the study.

• If so, the study should be rejected and another one found to help answer the clinical question. This requires clinical judgment to make this call.  Contact your clinical librarian for assistance in searching for articles or critically appraising them!

Important places to look for bias and confounding:

• Intention to Treat vs Per Protocol follow up
  • ITT - once you are randomized you are analyzed in the group you were randomized to. It doesn’t matter if you didn’t get the assigned “treatment”. Your outcomes still count toward this group. The main reason this is done is to preserve the benefit of randomization (equalizing confounders)
  • Per-protocol analysis - your outcomes count in the group to which “treatment” you actually received
• Bias in Study Design:
  • Selection bias : Participants in research may differ systematically from the population of interest
  • Allocation bias: Researchers know or predict which intervention the next eligible participant is supposed to receive.
• Bias in Funding
  • AKA sponsorship bias - refers to the tendency of a scientific study to support the interests of the study's financial sponsor.
  • This could be done through posing a research question such that the answer is true but misleading; choosing unrepresentative study populations; administering a competitors drug at a non-optimal dose (in comparator trials); questionable choices made while analyzing the data; non-publication of statistically nonsignificant results; selective reporting of outcomes; and multiple publication of positive results.
  • This is a huge concern in RCTs/medical literature as many studies are funded by pharmaceutical companies, and due to publication bias
• Bias in Data Collection:
  • Ascertainment bias -  More extensive data collection on some members of the target population than others.                                      
  • Attrition bias: Different rates of loss from the treated/exposed groups vs the untreated or unexposed individuals
  • Recall bias - participants do not remember previous events or experiences accurately or omit details: the accuracy and volume of memories may be influenced by subsequent events and experiences. Recall bias is a problem in studies that use self-reporting such as case controls or retrospective cohorts. 
  • Measurement bias-  errors in measuring exposure or disease such that different groups in the study are not assessed similarly. This results in differential misclassification (errors in how exposure or disease groups are classified or labeled).
    • Differential misclassification usually leads to overestimation of the effect of the exposure (or treatment).
  • Length-Time Bias -  is an overestimation of survival duration due to the relative excess of cases detected that are asymptomatically slowly progressing, while fast progressing cases are detected after giving symptoms.
  • Lead Time Bias -  phenomenon where early diagnosis of a disease falsely makes it look like people are surviving longer. This occurs most frequently in the context of screening.
    • For example, a man with metastatic lung cancer dies at age 70. His cancer was discovered 1 year ago, when he was 69. Therefore, it appears as if he lived for 1 year with the cancer. However, imagine that instead his cancer was discovered on a screening CT scan when he was 65 years old. If he still dies at the age of 70, it now looks like he survived for 5 years with the diagnosis of cancer (the 5 year survival rate is much better), but in fact there was no real change in his survival.
• Bias in Study Performance:
  • Observer bias – if any of the portion of data involves observation by a person 
• Bias in Study Reporting:
  • Reporting Bias - Studies showing no effect/not positive effects not being reported
    • The failure to publish the results of a study “on the basis of the direction or strength of the study findings.”
      • “Paxil for children adolescents : “…GSK conducted at least five studies on the use of Paxil [paroxetine] in children and adolescents. However, GSK only published and disseminated one of these studies, which showed mixed results on efficacy. The lawsuit alleges that the company suppressed the negative results of the other studies, which failed to demonstrate that Paxil is effective and which suggested a possible increased risk of suicidal thinking and acts” . “In fact there was no evidence of paroxetine efficacy for teenagers and a small but real increased suicide risk”
  • Studies report composite outcome to achieve statistical significance rather than intended primary outcome
  • Immortal Time Bas - period of cohort follow-up time during which death (or an outcome that determines end of follow-up) cannot occur.