Clinical Data Can Mislead Without Meaningful Analysis

Tony Hagen @oncobiz
Published: Tuesday, Jun 27, 2017
Bobby Green, MD

Bobby Green, MD

Good data and data analysis can tell doctors a lot about their performance that they may have overlooked, said Bobby Green, MD, senior vice president for clinical oncology at Flatiron Health in New York City. Without careful analysis, doctors may assume they’re doing the right thing even though the truth may be different. “I think I’m a good oncologist,” he said. “I know my patients think I’m a good oncologist. My referring docs think I’m a good oncologist, but the reality is I don’t really have any idea whether I’m a good oncologist.” The difference is being able to prove it by the numbers, he said.

CMS has put so much emphasis on reported data that it is now paramount for oncology practices to understand how data should be sifted and interpreted to arrive at meaningful conclusions, said Green, who in addition to his work for Flatiron, continues to work as a practicing oncologist 1 day a week. He spoke in April at the 2017 Community Oncology Alliance annual meeting.

At Flatiron, Green is working on ways to mine healthcare information for clues to better outcomes and lower costs. Through his clinical work, he is participating in the Oncology Care Model (OCM) pilot for better outcomes and lower costs. “We’re being asked to look at several different things,” he said. “We’re being asked to measure the quality and cost within our practices. We’re being asked to make interventions in our practices that have the potential to affect care and the potential to affect the financial viability of our practices.”

In his presentation, Green described numerous situations where cost savings and quality initiatives based on Big Data can go awry without careful selection and handling of data points. The Medicare Access and CHIP Reauthorization Act (MACRA), the Medicare reform law, has been CMS’ tool to spur medical practices to gather data and report on a number of performance measures, and the analyses based on this data will determine how well practices are remunerated. Therefore, oncology practice managers and physicians must understand how to correctly interpret the data they’re gathering and that are coming to them from CMS, Green said. He noted that under the CMS plan, by 2020, 60% of the physician payment formula will be based on performance as measured from reported data.

Good measurement begins with a sound analysis plan and continues with a willingness to look deeper than surface impressions, he said. “The first answer you get is not always the right answer.” For an example, he reviewed a Flatiron study of EGFR and ALK testing in cases of squamous non–small cell lung cancer (NSCLC). Ideally, about 20% of patients with squamous NSCLC would be tested for EGFR and ALK, particularly, those with a lifetime history of smoking or for whom inadequate tissue samples were obtained. The Flatiron study of 4095 patients and 206 clinics showed an average testing rate of 21%, which Green said was “spot on.” But on closer inspection, it was discovered that some of the clinics had been testing far fewer than 10% of patients with squamous NSCLC, whereas others had been testing close to 100%, both of which were clearly undesirable extremes, Green pointed out.

This example shows the importance of looking at data more closely to avoid being misled by superficial observations, he said. “If you drill down, you actually see some discrepancies, and I would argue that these are discrepancies that if you’re a practice that’s either testing too many or not testing enough patients, this is an improvement that you would want to make.”

Conversely, an initial examination of data can tell you you’re doing the wrong thing and only upon closer inspection are you able to determine that things are being done correctly. To demonstrate, Green presented a study of KRAS testing in patients with metastatic colorectal cancer (mCRC) who were potential candidates for EGFR inhibitor therapy. He noted that KRAS testing is advised for mCRC because it’s harmful to give EGFR inhibitor therapy to patients who are KRAS-positive. An initial look at the data showed KRAS testing rates peaking at 71% of patients (n = 168) in 2012 and dropping to 57% (n = 748) in 2014, neither of which was acceptable, Green said.

View Conference Coverage
Online CME Activities
TitleExpiration DateCME Credits
Community Practice Connections™: Addressing Post-Transplant Obstacles: Current and Emerging Strategies to Evolve the Standard of Care for Patients With Graft-Versus-Host DiseaseMar 28, 20192.0
2017 Year in Review™: Clinical Impact of Immunotherapies in the Treatment of CancerMar 30, 20191.75
Publication Bottom Border
Border Publication