Good Health Policy Requires High-Quality Evidence. Let’s Remember That With Drug Pricing.

Am J Manag Care. 2022;28(9):427-428. https://doi.org/10.37765/ajmc.2022.89218

Takeaway Points

• Good policy requires good research.
• The lack of reproducibility in research results raises questions about the quality of evidence.
• We should hold studies informing policy to the highest possible standards. Not doing so means our policy debate is, at a minimum, ill-informed. At worst, a blind acceptance of misleading information could lead to policies that will hurt patients, consumers, and employers.
• The National Pharmaceutical Council recently released Health Care Spending Guiding Principles to help foster a rigorous, evidence-based evaluation and discussion of health care spending estimates and policies.
• Our aim should be to improve health care spending efficiency and maximize patient health. Using such principles to determine the true rigor of any study gives policy makers, journalists, and others better evaluation tools in that effort.
• Good research, placed in the appropriate context and fully understanding the inherent risk, uncertainty, and potential trade-offs of any recommendations, is incredibly valuable.

Health spending continues to rise, with 2020 total spending exceeding $4.1 trillion, or 19.7% of US gross domestic product.¹ Concerns are growing regarding the sustainability of continued increases, and the details of the latest proposal to allow the federal government to “negotiate” drug prices were just released.² At this pivotal moment in the health spending debate, what policy makers need most is an honest, realistic, and evidence-based discussion. That’s not happening, mainly because policy makers are inundated with poorly designed studies, with conclusions based on unrealistic assumptions not supported by evidence.

With the right tools, policy makers and other decision makers can better evaluate such studies to help ensure that policy conversations are appropriately grounded in evidence. Quality research allows them to identify shortcomings in rigor and methodology; it also can foster greater trust in research that is transparent in methodology and realistic in assumptions.

That puts a particular burden on health policy researchers, who should be designing studies that support assertions using data, assumptions, and methodologies appropriate for the issue being analyzed and transparent about which questions they are intended to provide pertinent insight into.

Unfortunately, many studies in the public arena fall far short of this. Reproducibility, an independent researcher’s ability to replicate a study’s results, is one standard with which to evaluate evidence quality. Three studies assessing reproducibility found that this standard was only met between 40% and 62% of the time, which raises questions about the quality of evidence.^3-5

Challenges are not limited to the quality of the results but also lie in the interpretation of these results. An oft-cited 2015 study, for example, notes that “many of the key insights” that led to transformative medicines first began with publicly funded basic research.⁶ Supporters of government price controls have said the findings of this study show that as long as government investment is substantial, the pace of innovation will remain unchanged.⁷

However, policy makers ought to know that the study left out the fact that the drug industry may invest more than $1 billion per therapy compared with just tens of millions of dollars of government investment. If private investment declines, what makes up the gap? The study is silent on that question.

The point here is that we should hold studies informing policy to the highest possible standards. Not doing so means our policy debate is, at a minimum, ill-informed. At worst, a blind acceptance of misleading information could lead to policies that will hurt patients, consumers, and employers.

As part of our commitment to rigorous and quality research, the National Pharmaceutical Council recently released Health Care Spending Guiding Principles to help foster a rigorous, evidence-based evaluation and discussion of health care spending estimates and policies.⁸

These principles can help decision makers evaluate the methodological rigor, limitations, and alignment with patient-centered care and provide a framework to guide the evaluation of policy proposals through a better process of identifying the trade-offs, uncertainty, and risk of unintended consequences for these proposals.

Our aim should be to improve health care spending efficiency and maximize patient health. Using such principles to determine the true rigor of any study gives policy makers, journalists, and others better evaluation tools in that effort.

Good research, placed in the appropriate context and fully understanding the inherent risk, uncertainty, and potential trade-offs of any recommendations, is incredibly valuable. Everyone participating in the drug pricing debate, no matter what their views, should welcome a good research standard.

Doing so will allow stakeholders across the health care system to have an honest discussion about how to best address rising health care costs and foster patient-centered care.

That’s a conversation worth having. 

Author Affiliations: National Pharmaceutical Council (MC, JMO), Washington, DC.

Source of Funding: None.

Author Disclosures: Mr Ciarametaro is a former employee of the National Pharmaceutical Council, a health policy research organization supported by the biopharmaceutical industry. Dr O’Brien is an employee of the National Pharmaceutical Council.

Authorship Information: Concept and design (MC, JMO); acquisition of data (MC); analysis and interpretation of data (MC, JMO); drafting of the manuscript (MC, JMO); critical revision of the manuscript for important intellectual content (MC, JMO); obtaining funding (MC, JMO); administrative, technical, or logistic support (MC, JMO); and supervision (MC, JMO).

Address Correspondence to: John Michael O’Brien, PharmD, MPH, National Pharmaceutical Council, 1717 Pennsylvania Ave NW, Ste 800, Washington, DC 20006. Email: jmob@npcnow.org.

REFERENCES

1. Kurani N, Ortaliza J, Wager E, Fox L, Amin K. How has U.S. spending on healthcare changed over time? Peterson-KFF Health System Tracker. February 25, 2022. Accessed July 8, 2022. https://www.healthsystemtracker.org/chart-collection/u-s-spending-healthcare-changed-time/#Total%20national%20health%20expenditures,%20US%20$%20Billions,%201970-2020

2. Subtitle I—prescription drug pricing reform. Senate Finance Committee. Accessed August 1, 2022. https://www.finance.senate.gov/imo/media/doc/070622%20Prescription%20Drug%20Pricing%20Reform%20Leg%20Text.pdf

3. Open Science Collaboration. Psychology. Estimating the reproducibility of psychological science. Science. 2015;349(6251):aac4716. doi:10.1126/science.aac4716

4. Klein RA, Vianello M, Hasselman F, et al. Many Labs 2: investigating variation in replicability across samples and settings. Adv Methods Pract Psychol Sci. 2018;1(4):443-490. doi:10.1177/2515245918810225

5. Camerer CF, Dreber A, Holzmeister F, et al. Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nat Hum Behav. 2018;2(9):637-644. doi:10.1038/s41562-018-0399-z

6. Kesselheim AS, Tan YT, Avorn J. The roles of academia, rare diseases, and repurposing in the development of the most transformative drugs. Health Aff (Millwood). 2015;34(2):286-293. doi:10.1377/hlthaff.2014.1038

7. Kesselheim AS, Avorn J. Letting the government negotiate drug prices won’t hurt innovation. Washington Post. September 22, 2021. Accessed June 13, 2022. https://www.washingtonpost.com/outlook/2021/09/22/drug-pricing-negotiation-biden-bill/

8. Health Care Spending Guiding Principles. National Pharmaceutical Council. May 12, 2022. Accessed May 12, 2022. https://www.npcnow.org/hcsgp