Recognize what is (and isn’t) captured in RWD

This piece is the second in a series on avoiding common pitfalls when using real-world data (RWD). In our first blog, we saw how crucial it is to select data that is fit for purpose, aligning key elements in the data asset to the needs of your business questions. We also learned the importance of being intentional and coming into the data with a guiding question to help you avoid getting lost in the weeds.

In this edition, we’ll dive into the challenges researchers face when their expectations of the information captured in RWD don’t match reality.

Data aren’t collected with researchers in mind

Have you ever felt like the elements you expected to see didn’t appear in the data? Or have you been in a situation where the treatment approaches you thought you’d observe don’t match what’s actually captured?

The truth is, clinical practice in the real-world setting is much messier than what’s spelled out in evidence-based guidelines. As a result, the data that providers collect reflect all those variances, adding complications to the already difficult job of a life sciences researcher.

To better understand how RWD functions and maximize your data investment, review 3 common pitfalls and our recommendations to overcome them below.

Pitfall 3: Assuming medicine is practiced like a clinical trial

Although we all understand that a randomized, controlled clinical trial isn’t meant to perfectly reflect “real life,” researchers often forget this when reviewing a real-world data set. Consequently, researchers may be surprised when they don’t see specific elements captured in RWD as they were in clinical studies.

Rather than viewing this absence as a shortcoming of the data, consider that clinicians in real-world practice may not be performing certain tests or assessments in a routine matter.

In addition, outcomes of interest in RWD may be less well-defined than they are in clinical trials, given the ongoing, multifactorial nature of patient care and the decreased emphasis on standardized data collection.

Researchers should expect that some diagnostic tests, outcomes and treatment details may be neither as complete nor as curated as they would prefer.

Recommendations for working effectively with real-world clinical data:

Recognize that the limitations described above do not make the data unusable.

Get creative. Can you develop composite scores or other proxies for the desired outcomes or patient characteristics?

• For instance, pain or functionality scores may not show up in RWD because they are too complicated or time-consuming for clinicians to measure and record.
• Instead, you may encounter more general qualitative descriptors, such as mild, severe, improving or worsening.
• And in the absence of clinical notes, ask yourself, “What do I have access to?” Then consider, “Is that a suitable proxy for what I am trying to understand?”

Pitfall 4: Thinking claims data capture all patient interactions

If researchers were the ones designing the information-gathering mechanisms in health care, it’s fair to say they wouldn’t structure them like our current claims system.

Claims are meant to communicate the minimum information necessary to support reimbursement for health services. They may also be incomplete at times as patients move from one health plan to another, leaving gaps in care timelines.

For a while, claims were the only consistent data we had to conduct analyses on, so we made it work. But claims miss rich, meaningful details that could better inform analyses and help uncover insights for life sciences companies.

We can’t say the industry hasn’t tried to make things better. There have been attempts to support more thorough claims documentation, though the reality isn’t where we want it to be. For instance, providers can report social determinants via Z codes, but a recent government review showed that they only appeared in 1.59% of Medicare beneficiary claims.*

Recommendations:

Understand the factors driving the presence and absence of certain information in the data.

• For example, open claims (sourced from a clearinghouse) may offer more longitudinality in a patient’s journey — but only if the patient seeks care from a provider who uses the same clearinghouse. There’s a layer of uncertainty that will always exist.

Recognize that sometimes what is not in the data can be just as valuable to a researcher as what is.

• Closed claims, otherwise known as eligibility-controlled claims, are sourced from health plans.
• And while they, too, will show when a patient seeks care or fills a prescription, they’ll also allow you to analyze when the patient doesn’t take action.

Lay out the key objectives of your RWD analyses ahead of time and line them up with the appropriate data source to meet your study needs.

• Thoughtfully considering the questions you’re seeking to answer is critical to selecting a fit-for-purpose data set.

Consider linking claims data with clinical data to provide better visibility into the full continuum of care.

• Linked data sets can help you connect the dots across patient health histories with information on utilization, adherence and the costs associated with care.
• It may also be helpful to verify your findings in linked data by revisiting the context behind the research in which the data were collected or reviewing comparable existing literature.

Pitfall 5: Misjudging real-world physician and patient behavior

Researchers shouldn’t expect treatment approaches reflected in RWD to match ideal care pathways. We don’t always follow evidence-based guidelines in our everyday lives. When is the last time you reached for a second serving of vegetables instead of a dessert? Human behavior is an imperfect thing. In the same way, variability in the way patients behave and physicians practice medicine is inevitable.

We can attribute this variation to a broad range of factors, including those that are clinical or behavioral in nature.

Such factors include but are not limited to:

• Physician decision-making and clinical judgment
• Patient financial concerns
• Patient culture and lifestyle
• Treatment side effects
• Concurrent conditions and treatment
• Insurance-related reasons

Recommendations:

Appreciate that care variation is real and is an understood driver of different costs and outcomes.

• Expect to see differences in treatment approaches for similar patients.

Be curious about why the variation may be occurring. It can help you learn about what is going on in the market.

• Consider using the observations within the data as a potential discussion point with other researchers and clinicians.

Seek to understand why certain care pathways are or are not followed, especially as we see our data sources expand to cover broader population segments.

• Sometimes this will mean analyzing what’s missing from the data just as closely as analyzing what’s there.

Take a multidisciplinary approach.

• Bringing all the experts to the table on a particular project will contribute to an evolving understanding of clinical decision-making and behavioral and economic factors that impact RWD. Don’t assume data scientists can fly solo on analyses, as many require additional context that can be substantiated by multiple perspectives.

Be thoughtful in your RWD analysis approach

By addressing the pitfalls outlined here, researchers can make the most of the information provided in RWD assets. For more on maximizing your RWD investment, read our first article in the series, “Using the right tools to approach real-world data.”

We can help you put evidence into action.

Request more information