Improving clinical trial design

Clinical trials are how we identify and test potentially life-altering medicines. Which medicines actually work, and how effective and safe are they? Historically, our ability to answer these questions was very limited, and relied on basic observations, case reports, doctors’ experiments—which often had little oversight or consistency—and occasionally, statistical analysis of hospital or mortality records, if they existed.

But in the 20th century, a new type of study was developed: the clinical trial, typically as a randomized controlled trial (RCT). In a RCT, participants are randomly assigned to receive either a medicine or a control. This simple concept has been revolutionary — a powerful source of evidence for understanding cause and effect — and can also help reduce biases from researchers and drug manufacturers, who may want to hype up their products.

Since then, clinical trials have become the benchmark for testing medicines before they’re approved.

This process brings benefits: doctors have higher-quality evidence on whether drugs are effective and safe, helping them prescribe better treatments; researchers can learn from clear evidence of successes and failures to develop new and improved drugs.

But it also comes with costs. One is the expense of running clinical trials (often reaching hundreds of millions of dollars), which discourages researchers from developing new, experimental drugs and drugs without a large financial market. Another is the time it takes to conduct trials (typically several years), which delays access to potentially life-saving drugs.

Over time, clinical trials have generally grown larger and more complex.

New trial designs, such as challenge trials, platform trials, and adaptive trials, have been developed to improve the efficiency and outcomes of trials and reduce expenses or timelines. Drug regulation has also evolved, with more oversight and transparency, along with new pathways to support drug development for specific markets, such as rare diseases, where it can be difficult to find enough participants.

By improving trial design, we can often reduce research waste, and potentially reduce the costs and timelines of running trials, allowing life-saving drugs to reach patients more quickly.

Unfortunately many of these new approaches have been limited so far to a few countries, such as the United States and the United Kingdom, and only been applied in a few areas of medical research. One important reason is a lack of people with the statistical training and expertise to apply these ideas to different fields and countries.

In this introductory syllabus, I provide an overview of readings on clinical trials: their history, regulation, and methods; data sources on clinical trials; and courses for training in this field, focusing on the design and analysis of clinical trials.

If you only have 5 hours, these readings and videos will give you an introduction to clinical trials and their impact in the world: from what makes them so important in medical research, to their history, evolution, and the basic methods and concepts in the field.

• Article: Why randomized controlled trials matter and the procedures that strengthen them by Saloni Dattani (that’s me) — I make the case that RCTs have transformed our understanding of cause and effect, and give an introduction to clinical trials and concepts in the field.
• Essay: The pharma industry from Paul Janssen to today: why drugs got harder to develop and what we can do about it. Alex Telford takes you through the history of drug testing in the United States, and why it can be so expensive and time-consuming to test drugs today.
• Podcast episode: Sir Martin Landray on saving a million lives. Martin Landray co-led one of the most important COVID-19 drug trial in the world, called the RECOVERY trial. This single trial helped test more than 10 drugs in two years, and rapidly identified drugs that are estimated to have saved over a million lives during the pandemic.
• Memos: The Case for Clinical Trial Abundance by the Institute for Progress and Renaissance Philanthropy. This set of short memos, written by experts such as Martin Landray and Stuart Buck, introduces several ideas for improving and speeding up clinical trials in the United States.
• Book chapter: Clinical Research and Drug Regulation (1993). This book chapter, which has a rather dull sounding title, covers an incredibly important period of drug regulation history: the AIDS epidemic. It highlights how social movements changed the process of regulation, and how high the stakes can be to identify effective and safe drugs quickly.

If you have more time, these readings, lectures, and databases will help you explore more of the field. These are not intended to give you full expertise in clinical trials, but rather an introduction to important areas and concepts in the field.

How did randomized controlled trials become the cornerstone of medical testing? What drove the creation of modern drug regulations, shaping how medicines are tested and approved?

Reading the history of clinical trials has given me an appreciation for the role they’ve played in improving our knowledge of drug efficacy and safety. Their history also highlights how trial designs and regulations have evolved over time — and can continue to evolve — to advance medical innovation and knowledge.

• A brief history of randomized controlled trials (2016) by Laura Bothwell et al.
  • A recent history of drug regulation in the United States (2021)
  • A history of the drug regulation in the UK (2020)
  • A history of drug regulation in the EU (2024)
• The Institute of Medicine’s book on developing drugs for rare diseases (2010)

How can we improve the precision and accuracy of the results of clinical trials? How do we design them more efficiently, to save time and reduce waste and costs?

In my view, statistical power and trial design are undervalued areas that can improve clinical trials.

I’ve started here with foundational concepts like power calculations, p-values, type I and II errors, and stopping rules, before introducing Bayesian statistical trials, which help researchers incorporate prior knowledge and update their understanding as more data comes in.

• A primer on clinical trials by Hilary Vernon. This primer gives an introduction to concepts and basics of clinical trials and drug approval in the United States.
• Short introductory videos on statistical concepts by Daniël Lakens, including frequentism, likelihood and bayesian statistics, p-values, type 1 and 2 errors, and sample size justification, or his introductory textbook, Improving Your Statistical Inferences.
• Modern Epidemiology by Kenneth J. Rothman, Sander Greenland, Timothy L. Lash. My go-to textbook for an introduction to research methods and epidemiology. Chapter 10 covers field methods including randomized controlled trials.

With statistical understanding and new approaches, scientists can improve the design and analysis of clinical trials. Of course, these approaches have to be aligned with the aims of the study and their feasibility.

One way to improve trials is by increasing statistical power, for example, by increasing the sample size, using more precise measurement tools, or focusing on populations where changes are easier to observe.

Another way is by using variant trial designs, which can also improve statistical power, help answer different questions or be more useful in certain settings, and can even be more efficient in helping us learn about treatments’ effects.

In crossover trials, for example, participants receive multiple treatments and act as their own control, which can reduce variability and reduce the number of participants needed. Challenge trials, which expose participants to a controlled infection, can help measure effects faster and more precisely, with smaller sample sizes.

Adaptive trials allow trials to be modified based on interim results, which can improve efficiency and resource use, and platform trials, which test multiple treatments together, can help streamline the processes of recruitment and operation, and compare different drugs or doses. In cluster trials, entire groups like schools, households, or hospitals are randomized instead of individuals, which makes them more useful for studying interventions with group-level effects, such as infection control measures.

Here, I’ve given a few introductory readings, and links to courses, blogs, and training materials by researchers on some of these concepts.

• Platform trials by Matthieu Roustit et al. — a review on how platform trials are used, their benefits, methodological issues, and regulation and funding models
• Adaptive trial designs by Alex Kaizer — an introductory lecture to adaptive trial designs, from a US FDA perspective
  • Recent innovations in adaptive trial designs: A review of design opportunities in translational research by Alexander M Kaizer et al.
• Strategic and scientific contributions of human challenge trials for vaccine development by Yara-Natalie Abo et al. — an overview of how human challenge trials can help in vaccine development, including specifics and examples, and how they can be improved in the future
• Multi-arm multi-stage (MAMS) platform trials by the MRC Clinical Trials Unit at UCL — an overview of the advantages of MAMS trials, with teaching material
• Cluster randomized trials by the NIH Pragmatic Trials Collaboratory
• Crossover trials by the Cochrane Training Handbook

• The Pennsylvania State University’s course on clinical trial design and analysis
• Handbook of statistical methods for randomized controlled trials by KyungMann Kim, Frank Bretz, Ying Kuen K. Cheung, Lisa V. Hampson
• Cochrane Training Handbook — focuses on how to systematically review and interpret trials, as well as introducing the concepts
• NIH’s Pragmatic Trials Collaboratory living textbook — covers pragmatic trials, used to test real-world effectiveness of interventions, including through cluster randomized trials
• MRC’s Clinical Trials Unit materials — materials on how to improve and speed up clinical trials, improve their implementation, research and communication
• Biostatistics for Biomedical Research by Frank Harrell
  • Bayesian clinical trial design course by Frank Harrell
  • Statistical rethinking course by Richard McElreath — lectures and materials on Bayesian statistics more broadly
• Pragmatic and Group-Randomized Trials in Public Health and Medicine by the NIH’s Office of Disease Prevention — an online course on designing and analyzing pragmatic and cluster trials

These guidelines are a framework for how to design trials, collect data, and analyze their results in a way that meets regulatory and scientific standards, in different regions.

This helps improve the reliability of outcomes and facilitates the regulatory and approval process across regions. Standardized guidelines can also help make trial results comparable, which is important when synthesizing evidence across studies in meta-analyses.

• ICH Guidelines (International Council for Harmonisation): The ICH creates international standards to make clinical trials safe, ethical, and consistent across countries. These standards are very helpful for companies and researchers who want to run trials in multiple regions.
• FDA Guidance Documents: The U.S. FDA publishes guidance on running clinical trials, with specific documents for different types of products like drugs and devices. These guides include everything from trial design to statistical requirements.
• EMA Guidelines: The European Medicines Agency issues guidelines similar to the FDA’s but tailored to EU standards. They also cover special cases, like trials for rare diseases, which can be trickier to design.
• WHO Guidelines for Clinical Trials: The World Health Organization provides advice on running trials around the world, with a focus on ethics and logistics in lower-income countries.
• CONSORT Statement: This is a set of guidelines for reporting the results of randomized trials. Clear reporting is key to making trial findings useful for doctors, researchers, and the public.

These databases help make information about trials accessible and transparent. They allow researchers to track ongoing studies, compare trial designs, and analyze results to generate evidence for the safety and efficacy of treatments. Some databases below track approved drugs and biological products.

• ClinicalTrials.gov: A central hub for tracking clinical trials from around the world, supported by governments, private companies, and nonprofits. It shows you what studies are underway and what their results are, providing insight into how and why different trial designs are used.
• Cochrane Library: Cochrane pulls together results from lots of trials to create big-picture reviews. These reviews provide evidence on how well treatments work and how safe they are — ideal for when you need an overview of evidence on a medical intervention.
• European Union Clinical Trials Register (EUCTR) and EudraCT: These two databases offer data on clinical trials in the EU. EUCTR is the public-facing registry for EU clinical trials, showing protocols and results, while EudraCT is a larger, more detailed repository mainly used by EU regulators and researchers.
• FDA Drug Trials Snapshots: This U.S. resource offers insights into trial demographics for approved drugs, showing how outcomes may vary by race, age, and gender.
• Drugs@FDA: This is the U.S. FDA’s main database of approved drug products. It has everything from a drug’s approval history to its uses in treating different diseases.
• FDA’s Orange and Purple Books: These two lists cover FDA-approved drugs and biologics (drugs made from living cells). The Orange Book lists approved drugs, while the Purple Book lists licensed biological products and similar products.
• PubMed/MEDLINE and Embase: These two databases that store biomedical and clinical research. PubMed is useful for general biomedical literature, while Embase covers drug-related research in particular and often includes studies not listed in PubMed.

Here are some people to follow on social media and blogs to learn more about clinical trials:

• Andrew Althouse
• Frank Harrell
• Hilda Bastian
• Natalie Dean
• Solomon Kurz

Get in touch if you have ideas for how to expand this syllabus! My email address is saloni@scientificdiscovery.dev