I’m J Manag Care. 2022; 28 (9): In press
- Good policy requires good research.
- The lack of reproducibility in research results raises questions about the quality of the evidence.
- We must conduct research that informs policy to the highest possible standards. Failure to do so means that our policy debate is, to say the least, ill-informed. At worst, a blind acceptance of misleading information can lead to policies that will harm patients, consumers, and employers.
- The National Pharmaceutical Council recently published the Health Care Expenditure Guiding Principles to help foster rigorous, evidence-based evaluation and discussion of health care spending estimates and policies.
- Our goal should be to improve the efficiency of health care spending and maximize patient health. Using such principles to determine the true rigor of any study gives policymakers, journalists, and others better tools of evaluation in that endeavor.
- Good research, placed in the right context and fully understanding the inherent risk, uncertainty and potential trade-offs of any recommendation, is incredibly valuable.
Health care spending continues to rise, with total 2020 spending exceeding $4.1 trillion, or 19.7% of US gross domestic product.1 Concerns are growing about the sustainability of the ongoing increases and details of the latest proposal to allow the federal government to “negotiate” drug prices have just been released.2 At this pivotal moment in the health spending debate, what policymakers need most is an honest, realistic, and evidence-based discussion. This is not happening, largely because policymakers are inundated with poorly designed studies, with conclusions based on unrealistic assumptions that are not supported by evidence.
With the right tools, policymakers and other decision makers can better evaluate such studies to help ensure that policy conversations are based on appropriate evidence. Qualitative research allows them to identify shortcomings in rigor and methodology; it can also foster greater confidence in research that is transparent in methodology and realistic in assumptions.
This places a particular burden on health policy researchers, who must design studies that support claims using data, assumptions, and methodologies appropriate to the issue being analyzed and transparent about which questions they aim to provide relevant insight.
Unfortunately, many studies in the public arena are far from this. Reproducibility, the ability of an independent researcher to replicate the results of a study, is a standard by which the quality of evidence is assessed. Three studies that assessed reproducibility found that this standard was met only between 40% and 62% of the time, raising questions about the quality of the evidence.3-5
The challenges are not only limited to the quality of the results, but also lie in the interpretation of these results. An oft-cited 2015 study, for example, notes that “many of the key insights” that led to transformative medicines first began with publicly funded basic research.6 Supporters of government price controls have said that the findings of this study show that as long as government investment is substantial, the pace of innovation will remain unchanged.7
However, policymakers should be aware that the study left out the fact that the drug industry can invest more than $1 billion per therapy compared to only tens of millions of dollars of government investment. If private investment falls, what accounts for the gap? The study is silent on this question.
The point here is that we need to do research that informs policy to the highest possible standards. Failure to do so means that our policy debate is, to say the least, ill-informed. At worst, a blind acceptance of misleading information can lead to policies that will harm patients, consumers, and employers.
As part of our commitment to rigorous, quality research, the National Pharmaceutical Council recently published the Healthcare Expenditure Guiding Principles to help foster rigorous, evidence-based evaluation and discussion of health care spending estimates and policies. .8
These principles can help decision makers assess methodological rigor, limitations, and alignment with patient-centered care and provide a framework to guide the evaluation of policy proposals through a better process of identifying trade-offs, uncertainty, and risk of consequences. unintentional for these propositions.
Our goal should be to improve the efficiency of health care spending and maximize patient health. Using such principles to determine the true rigor of any study gives policymakers, journalists, and others better tools of evaluation in that endeavor.
Good research, placed in the right context and fully understanding the inherent risk, uncertainty and potential trade-offs of any recommendation, is incredibly valuable. All participants in the drug pricing debate, regardless of their views, should welcome a good standard of research.
Doing so will allow stakeholders across the health care system to have an honest discussion about how best to address rising health care costs and promote patient-centered care.
It’s a conversation worth having.
Author Affiliation: National Pharmaceutical Council (MC, JMO), Washington, DC.
Funding source: None.
Author’s disclosures: Mr. Ciarametaro is a former employee of the National Pharmaceutical Council, a health policy research organization supported by the biopharmaceutical industry. Dr O’Brien is an employee of the National Pharmaceutical Council.
Copyright information: Concept and design (MC, JMO); data acquisition (MC); data analysis and interpretation (MC, JMO); drafting the manuscript (MC, JMO); critical revision of the manuscript for important intellectual content (MC, JMO); receiving funds (MC, JMO); administrative, technical or logistical support (MC, JMO); and supervision (MC, JMO).
Correspondence address to: John Michael O’Brien, PharmD, MPH, National Pharmaceutical Council, 1717 Pennsylvania Ave NW, Ste 800, Washington, DC 20006. Email: firstname.lastname@example.org.
1. Kurani N, Ortaliza J, Wager E, Fox L, Amin K. How has US health care spending changed over time? Peterson-KFF Health System Tracker. February 25, 2022. Accessed July 8, 2022. https://www.healthsystemtracker.org/chart-collection/us-spending-healthcare-changed-time/#Total%20national%20health%20expenditures,%20US%20$% 20 billion,% 201970-2020
2. Subtitle I-Prescription Drug Pricing Reform. Senate Finance Committee. Accessed August 1, 2022. https://www.finance.senate.gov/imo/media/doc/070622%20Prescription%20Drug%20Pricing%20Reform%20Leg%20Text.pdf
3. Open scientific cooperation. Psychology. Assessing the reproducibility of psychological science. science. 2015;349(6251):aac4716. doi:10.1126/science.aac4716
4. Klein RA, Vianello M, Hasselman F, et al. Multiple Laboratories 2: Investigating variation in repeatability between samples and settings. Methods Adv Pract Psychol Sci. 2018; 1 (4): 443-490. doi: 10.1177/2515245918810225
5. Camerer CF, Dreber A, Holzmeister F, et al. Assessing the replicability of social science experiments in Nature AND science between 2010 and 2015. Nat Hum Behav. 2018; 2 (9): 637-644. doi: 10.1038/s41562-018-0399-z
6. Kesselheim AS, Tan YT, Avorn J. The roles of academia, rare diseases, and repurposing in the development of more transformative drugs. Health Aff (Millwood). 2015; 34 (2): 286-293. doi:10.1377/hlthaff.2014.1038
7. Kesselheim AS, Avorn J. Letting government negotiate drug prices won’t hurt innovation. Washington Post. September 22, 2021. Accessed June 13, 2022. https://www.washingtonpost.com/outlook/2021/09/22/drug-pricing-negotiation-biden-bill/
8. Guiding principles for health care costs. National Pharmaceutical Council. May 12, 2022. Accessed May 12, 2022. https://www.npcnow.org/hcsgp