Recruiting for Your Pragmatic Clinical Study
3. Healthplan Data: A Treasure Trove for Your Pragmatic Study, Part 1
Because it covers the care continuum for huge populations, healthplan data offers strong advantages for planning, recruiting for, and analyzing outcomes of your pragmatic clinical study. But it’s designed for payment—not studies. Do you know how to leverage its strengths while minimizing the ‘gotchas?’
Published on
January 11, 2019
"Cloquet hated reality, but realized it was still the only place to get a good steak." - Woody Allen

“How does a person’s insurance benefit plan affect their choice, adherence to, and outcomes of a therapeutic product or treatment pathway?”

“If the cost of treatment X is lower than Y, how do know whether that was due to improved outcomes or using lower-cost services?”

“How can I efficiently recruit for and do a study that provides actionable insights on a wide variety of patients and doctors all over the country?”

Welcome back to TeloChain’s Real-World Healthcare Insights, where we explore how to discover the value in healthcare therapeutics - pharmaceuticals and medical devices - through real-world clinical studies. This series offers a deeper dive into pragmatic (real-world) clinical study design, especially the widely-distributed pragmatic study (WDPS), with its potential for yielding actionable insights into therapeutics in a wide range of clinical, patient, and healthcare services scenarios. Orientation to the series: HERE. Comparisons of study designs: HERE.

Cut to the punchline: Healthplan data yields a comprehensive picture of the continuum of care for very large, dispersed insured populations. But, it’s designed for payment, not for clinical outcomes studies. Yet in the hands of experts it can be remarkably useful for recruiting and measuring utilization and financial outcomes, especially in a widely-distributed pragmatic study.

In the last episode we looked at the interaction of what you want to find out (your questions) and how you go about doing that (study design). While giving the prospective randomized controlled trial (PRCT) its well-deserved due, we pointed out the PRCT’s limitations for understanding--and taking action in--the messy real world. We also noted the emergence of the pragmatic clinical study design as a way to gain actionable marketing and clinical insights into the real world use and outcomes of treatments across a wide variety of clinical scenarios, patients, clinicians, and healthcare settings.

Today we dive into the pragmatic study design--especially a promising form called the widely-distributed pragmatic study (WDPS)--and data--healthplan demographics and claims--that in the right hands can support large-scale WDPS. We’ll look at healthplan data’s advantages and disadvantages (compared with other types of data) and offer expert tips on how to use it well for recruiting and as part of outcomes assessment.

Does the following sound like the real world to you? The PRCT tells you whether (2) and how well the treatment ‘worked’ on people who met the study’s inclusion and exclusion criteria, volunteered to be randomized, were treated by experienced physicians, monitored carefully, and usually were highly adherent.  

Or is it more like this? Patients don’t always fit the study’s inclusion or exclusion criteria; they are treated in settings from solo practices to multispecialty practices; their physicians’ advice is influenced by their training, local practices, what they remember from conferences, reading, conversations with colleagues and drug reps (and personal beliefs about what patients need and want); patients are influenced by their physicians, family, friends, reading, media, and personal beliefs and preferences; and healthcare delivery is influenced by availability of expert consultation, quality management systems, availability and use of clinical decision support and incentives (3). These factors may explain why even the most careful observational studies rarely reproduce the PRCT findings.

Who do we seek insights on in pragmatic studies?

Let’s take a closer look at a diagram from Episode 2: The green area 1 represents people who are actually using the treatment of interest. The blue area 3 represents people who exactly fit all of the clinical trial’s inclusion and exclusion criteria. The red area 4 represents people who shouldn’t be using the treatment because they won’t benefit or may be harmed. Note that this includes some people who meet the PRCT’s criteria! By definition, the PRCT can’t give us insights about the treatment for people who use the drug but don’t meet the PRCT criteria; and it might not tell us enough about the area 4 people (who meet the PRCT criteria but shouldn’t use the treatment). It would also be nice to understand the area 3 people, who the study found the treatment would help, but don’t use it (though they may be on a different treatment).

The real world is messier than clinical trials. 1: People who are both using and would benefit from the therapy. 2: People who meet the PRCT criteria and are both using and would benefit from the therapy. 3: People who meet the PRCT criteria and would benefit from the therapy, but aren’t using it. 4: People who meet the PRCT criteria, are using the therapy, but shouldn’t be (won’t benefit or are at  risk of harm)

Pragmatic studies evaluate treatments in real-world settings under relaxed conditions. For example, doctors and patients might change or modify treatment pathways, they may acquire or improve control of other diseases, and patients may be permitted to have variable adherence. Pragmatic studies can take a range of designs:

  • Prospective randomized pragmatic clinical trials (PRPCT): Run in patient care settings, often in large, well-managed primary care or multispecialty groups. The NIH offers excellent resources for establishing a PRPCT, as well as a catalog of existing or planned trials (4)
  • Prospective pragmatic observational studies (PPOS): Don’t randomize, but can yield valuable insights about effectiveness if carefully designed. We might observe, for example, the introduction of a treatment at some clinics while others continue to use a different treatment (a natural experiment).  May help healthcare systems decide when or how to implement new treatments or clinical programs (such as quality improvement or condition management)

Widely-distributed pragmatic studies (WDPS)--observational or randomized--are emerging as a way to gain insights into the use, outcomes and value of one or more treatments across a wide gamut of clinical scenarios, patient characteristics, and healthcare settings. If properly planned, executed and analyzed, WDPS has the potential for deepening our ability to deliver personalized healthcare.

Why is recruiting and study management different for WDPS?

The core idea of the WDPS is to take as representative a ‘biopsy’ of the wide range of patients, relevant clinical scenarios and healthcare settings as possible. This is best served with data that reflects the entire continuum of care over some continuous time frame, on lots of individuals with a wide variety of demographics and comorbidities, in a spectrum of care settings. Healthplan (insurance) data has historically been used to support retrospective observational studies to understand market dynamics, treatment patterns and outcomes. The strengths and limitations of healthplan data are well known and--in the hands of experts--can yield actionable insights.

Likewise, healthplan data is a powerful and practical starting-point in WDPS for recruiting and a valuable contributor to outcomes assessment—IF you know how to use it well.

To understand why, let’s take a look at what makes up healthplan data and its potential uses for recruiting (identifying individuals--and the physicians who treat them--who may be good candidates for a pragmatic study based on that study’s inclusion and exclusion criteria), and as a component of measuring the outcomes of your study.

What's in healthplan data? (5)

What are the opportunities, strengths, limitations and ‘need for expertise’ in healthplan data (and working with healthplans) in pragmatic studies? Find out in our next episode!

Want to know more? Find us HERE!


  1. According to Quote Investigator (, there is no substantive evidence that Allen pinched the line from Groucho Marx, stating that it first appeared in a 1977 story in The New Yorker.'
  2. To a specified degree of confidence that positive results were actually true and that if there really was a positive outcome, the study didn’t miss it.
  3. For example, an accountable care organization (ACO) is incented to deploy, use and improve systems to monitor and improve quality, resource utilization and cost. E.g. see Feldstein AC, et al. Effect of a patient panel-support tool on care delivery. Am J Manag Care. 2010;16(10):e256-66.
  4. Rethinking Clinical Trials is the NIH Collaboratory Living Textbook of pragmatic clinical trials (
  5. Of course, this is what the healthplan or certain of its permitted partners can see
  6. Interpreting dollar values on claims can be tricky. Claims may include: Billed or charged; allowed (negotiated payment--from all sources--for a specific service); reimbursed (usually same as allowed, but not always); paid (amount paid by the healthplan); member cost share (amount to be paid by member based on unmet deductibles, copayments and coinsurance); coordination-of-benefits (amount paid by another insurer, if any)
  7. Healthplan data doesn’t contain actual genomic data but clinical scenarios can sometimes be inferred from presence of claims for lab testing
  8. Selection bias (the potential tendency of volunteers to behave differently with respect to their health) comes from various sources and can be quite nuanced. While randomization neutralizes selection bias between its comparison groups, it can’t take into account the effect of volunteering to be randomized. The whole topic deserves more attention than we can give here. Let us know in the comments section if you’d like to hear more.
Recruiting for Your Pragmatic Clinical Study