On this episode of On the Evidence, Temitope Ojo and Katlyn Lee Milless discuss their experience with the Mathematica Summer Fellowship in 2021. Ojo is a doctoral candidate at the NYU School of Global Public Health. Her research focuses on implementation science as well as the prevention and management of cardiovascular disease and other non-communicable diseases in a global setting.

Milless is a doctoral candidate in basic and applied social psychology at the Graduate Center of The City University of New York. Her research takes a psychological approach to understanding how to promote gender and racial equity in education spaces, particularly among college students in STEM—science, technology, engineering, and mathematics.

On the episode, Ojo and Milless discuss their career paths, why they’re interested in applied research, and what they plan to do after completing their dissertations.

Find a full transcript of the episode here: https://www.mathematica.org/blogs/a-conversation-with-mathematicas-2021-summer-fellows

Mathematica is currently accepting applications for its 2022 Summer Fellowship program through the end of February. Find more information about the fellowship here: https://www.mathematica.org/career-opportunities/summer-fellowships

70 | Preventing Cardiovascular Disease and Promoting Equity in Higher Ed

On this episode of On the Evidence, Mathematica’s Mariel Finucane and John Deke join Tim Day of the Center for Medicare & Medicaid Innovation to discuss the application of evidence-informed Bayesian methods that not only confirm whether a policy or program works, but for whom.

Learn more about Mathematica's work using evidence-based Bayesian methods in applied policy research: https://mathematica.org/features/bayesian-methods

Read a brief about using a Bayesian framework for interpreting findings from impact evaluations prepared by Mariel Finucane and John Deke for the Office of Planning, Research and Evaluation at the Administration for Children and Families: mathematica.org/publications/moving-beyond-statistical-significance-the-basie-bayesian-interpretation-of-estimates-framework

Read a paper co-authored by Mariel Finucane that compares Bayesian methods with the traditional frequentist approach to estimate the effects of a Centers for Medicare & Medicaid Services demonstration on Medicare spending: mathematica.org/publications/revolutionizing-estimation-and-inference-for-program-evaluation-using-bayesian-methods

Read a paper co-authored by Tim Day describing an experiment to provide evidence that would be useful to policymakers and other decision makers through an interactive data visualization dashboard, presenting results from both frequentist and Bayesian analyses: https://www.researchgate.net/publication/335169870_Making_Evidence_Actionable_Interactive_Dashboards_Bayes_and_Health_Care_Innovation

Read Emily Oster’s newsletter article about why and how she applies Bayes’s Rule to interpret new evidence in the context of existing evidence, including a recent study (https://emilyoster.substack.com/p/does-pre-k-really-hurt-future-test) about the effects of a preschool program in Tennessee on future student test scores: https://emilyoster.substack.com/p/bayes-rule-is-my-faves-rule

72 | Bayesian Methods Could Provide the Key to Answering Which Policies Work Best for Whom

Across many safety net programs, workers with low income and their families face the threat of a sudden and unexpected loss of benefits if their earnings increase too much, sometimes resulting in a net decrease in overall income. Policymakers have long worried that the phenomenon, often described as the cash or benefits cliff, discourages work and reinforces dependence on public assistance. Over the past decade, the Social Security Administration has launched two national demonstrations intended to mitigate this so-called cash cliff effect in the Social Security Disability Insurance (SSDI) program.

The latest episode of On the Evidence explores the results of those demonstrations and what they mean for future efforts to address program cliffs in the SSDI program and many other safety net programs. Our guests for this episode are John Jones, David Wittenburg, and Diane Beaver.

Jones is an economist at the Social Security Administration in the Office of Research, Demonstration, and Employment Support who has overseen several large-scale randomized controlled trials testing potential changes to the SSDI program, including two discussed on this episode: the Promoting Opportunity Demonstration (POD) and the Benefit Offset National Demonstration (BOND).

Wittenburg is a senior fellow at Mathematica whose research on interventions to promote employment for people with disabilities includes evaluations of POD and BOND.

Beaver is an advisory services analyst at Mathematica who spent more than a decade at a community nonprofit counseling people on how work and other entitlements would affect their Social Security benefits. In that role, she played a part in implementing POD and has firsthand knowledge of what beneficiaries experience as they navigate the patchwork of program rules that govern the amount of government aid people can receive for housing, food, child care, health care, and other needs as their work status changes.

Find a full transcript of the episode here: mathematica.org/blogs/why-a-national-demonstration-to-mitigate-the-cash-cliff-in-one-safety-net-program-didnt-increase

Read the final evaluation report on POD: https://www.ssa.gov/disabilityresearch/documents/POD_Final_Evaluation_Report.pdf

Read the final evaluation report on BOND: https://www.ssa.gov/disabilityresearch/documents/BOND%20Deliv%2024e2%20FER%20Vol%201%2020181018.pdf

Find a summary of lessons learned from several decades of demonstrations by the Social Security Administration to test policy ideas in the Social Security Disability Insurance and Supplemental Security Income programs: https://www.ssa.gov/disabilityresearch/demonstrations/lessons.htm

Learn more about the Ultimate Demonstration referenced at the tail end of the episode: https://www.ssa.gov/disabilityresearch/documents/Simplification_Demo_TEP_Final_Report_Final%20Remediated.pdf

Watch a recorded discussion between Pamela Herd and Sebastian Jilke, professors at the McCourt School of Public Policy at Georgetown University, about administrative burden in the Social Security Disability Insurance program: https://www.ssab.gov/announcements/ssab-to-host-experts-on-researching-and-evaluating-equitable-access-to-social-security-programs/

78 | Addressing the Cash Cliff in Safety Net Programs: Lessons from a National Demonstration

In late May 2023, Mathematica coordinated with Congressman Derek Kilmer’s office to host a discussion on Capitol Hill about the impact of the Foundations for Evidence-Based Policymaking Act (Evidence Act) and how to further embed the use of data and evidence in federal decision making. This episode of On the Evidence features edited audio from the event with remarks from Kilmer and his House colleague, Rep. William Timmons, in addition to a panel discussion with experts from government, think tanks, research organizations, and academia. 

Kilmer represents Washington State’s Sixth District and was one of the Evidence Act’s original co-sponsors. He is also the sponsor of a new bipartisan resolution to establish a new commission on evidence-based policymaking that would advise Congress on how to make greater strides in embedding the use of data and evidence in federal agencies but also in Congress itself.

Timmons represents South Carolina’s Fourth District and is a co-sponsor of the resolution to establish a new evidence commission. 

The other speakers included Robert Groves, Nick Hart, Ruth Neild, and Susan Jenkins. 

Groves is a provost of Georgetown University and served as a member of the previous U.S. Commission on Evidence-Based Policymaking, whose recommendations informed the Evidence Act. 

Hart is the president and CEO of Data Foundation and served as policy and research director for the U.S. Commission on Evidence-Based Policymaking. 

Neild is an executive director of education and employment research at Mathematica and previously served as delegated director of the Institute of Education Sciences within the U.S. Department of Education. 

Jenkins directs the Division of Evidence, Evaluation and Data Policy at the U.S. Department of Health and Human Services, where she also serves as an evaluation officer.

A full transcript of the episode is available at mathematica.org/blogs/building-on-the-evidence-act-to-increase-the-positive-impact-of-federal-policymaking

Learn more about the U.S. Commission on Evidence-Based Policymaking, which made recommendations ahead of the Foundations for Evidence-Based Policymaking Act of 2018: https://obamawhitehouse.archives.gov/omb/management/commission_evidence

Read a fact sheet from the Bipartisan Policy Center on the Foundations for Evidence-Based Policymaking Act: https://bipartisanpolicy.org/blog/fact-sheet-foundations-for-evidence-based-policymaking-act/

Read a press release about the resolution to establish a new commission on evidence-based policymaking: https://kilmer.house.gov/media/press-releases/kilmer-spearheads-legislative-change-with-introduction-of-evidence-based-policymaking-resolution

Read the bipartisan resolution to establish a new commission on evidence-based policymaking to further embed the use of data and evidence in federal decision making: https://www.congress.gov/bill/118th-congress/house-concurrent-resolution/49/text?s=1&r=1&q=%7B%22search%22%3A%5B%22Evidence+commission%22%5D%7D

97 | Building on the Evidence Act

This episode of Mathematica’s On the Evidence podcast focuses on how research informs funding strategies to tackle an urgent social challenge. A couple of years ago, Mathematica reviewed evidence from across the world on whether education programs can prevent or mitigate violence and crime. USAID commissioned the report to help develop effective strategies for leveraging the education sector to address violence and crime in Latin America and the Caribbean. Although Mathematica’s researchers identified almost two dozen promising programs in the review, they also noted the need for further research. For example, few of the studies show causal impacts or provide evidence from an effective program in Latin America or the Caribbean.

To build on findings from Mathematica’s evidence review, USAID and the International Rescue Committee(IRC) partnered to launch an initiative called USAID Research in Education for Transformative Opportunities (RETO), which provides tailored evidence to decision makers in governmental and non-governmental institutions, such as local ministries of education. The initiative seeks to help those education leaders implement effective programs and policies in Northern Central America that will reduce local violence and crime.

The guests for this episode are Juan Carlos Rodríguez, Katie Appel, Melissa Chiappetta, and Emilie Bagby. Rodríguez and Appel are the director and deputy director, respectively, for USAID's RETO activity implemented by the IRC and its 12 national partners. Chiappetta is a senior education advisor with the Office of Regional Sustainable Development within USAID's Bureau for Latin America and the Caribbean. Bagby is a director at Mathematica, where she oversees our international education research.

Find the full transcript here: mathematica.org/blogs/developing-education-programs-to-prevent-violence-and-crime-in-latin-america-and-the-caribbean

Read the evidence review by Mathematica for USAID on the effect of education programs on violence, crime, and related outcomes: https://pdf.usaid.gov/pdf_docs/PA00XGXT.pdf

Learn more about RETO: https://www.youthpower.org/usaid-research-education-transformative-opportunities-reto

Learn more about IRC’s Airbel Impact Lab: https://airbel.rescue.org/?_ga=2.189623531.1744177215.1687884026-800337516.1664199528

Learn more about Mathematica’s evaluations for USAID’s Latin America and the Caribbean Reads (LAC Reads) initiative, which included the evidence review on education programs to prevent or mitigate violence and crime: https://staginginter.mathematica.net/projects/latin-america-and-the-caribbean-reading-evaluation

98 | Developing Education Programs to Prevent Violence and Crime in Latin America and the Caribbean

In May 2023, Mathematica hosted a convening on Capitol Hill about embedding evidence in federal decision making, with a focus on the legacy of the nearly five-year-old Foundations for Evidence-Based Policymaking Act, also known as the Evidence Act. One of the attendees that day was Robert Shea, an expert on performance improvement in government who served on the bipartisan U.S. Commission on Evidence-Based Policymaking, a group whose recommendations informed the Evidence Act. Shea is the chief executive officer for GovNavigators, a government management consulting firm, where he hosts a podcast called The GovNavigators Show. His career has also included posts at the White House Office of Management and Budget, the Senate Committee on Governmental Affairs, and the House Committee on Government Reform and Oversight.

On this episode of Mathematica’s On the Evidence podcast, Shea discusses the improvements he has witnessed since the mid-1990s in the use of data and evidence in federal decision making.

“We have seen a sea change in the ability of agencies to understand that they need to articulate what they’re trying to accomplish in terms of outcomes, report that transparently,” Shea says on the episode. “A lot of what we’re talking about today is sort of dig deeper—find out whether it’s what we’re doing that’s contributing to the ultimate outcome rather than some other factor.”

This episode is part of an occasional series on the show called Evidence in Government, which explores new developments in the halls of government and the role that evidence can play in decisions that could improve people’s lives. Mike Burns, Mathematica’s senior director of communications and public affairs at Mathematica, conducts the Evidence in Government interviews.

SHOW NOTES

Listen to The GovNavigators Show, a podcast Shea hosts with Adam Hughes about government management: https://www.govnavigators.com/podcast

Watch a recording of the event Mathematica hosted in May 2023 on the Evidence Act: https://www.mathematica.org/events/evidence-at-work-for-the-american-people

Learn more about the U.S. Commission on Evidence-Based Policymaking, which made recommendations ahead of the Foundations for Evidence-Based Policymaking Act of 2018: https://obamawhitehouse.archives.gov/omb/management/commission_evidence

Read a fact sheet from the Bipartisan Policy Center on the Foundations for Evidence-Based Policymaking Act: https://bipartisanpolicy.org/blog/fact-sheet-foundations-for-evidence-based-policymaking-act/

Read a press release about the resolution to establish a new commission on evidence-based policymaking: https://kilmer.house.gov/media/press-releases/kilmer-spearheads-legislative-change-with-introduction-of-evidence-based-policymaking-resolution

Read the bipartisan resolution to establish a new commission on evidence-based policymaking to further embed the use of data and evidence in federal decision making: https://www.congress.gov/bill/118th-congress/house-concurrent-resolution/49/text?s=1&r=1&q=%7B%22search%22%3A%5B%22Evidence+commission%22%5D%7D

100 | Robert Shea on the Federal Government’s Progress in Using Evidence

In this special feature, On the Evidence host J.B. Wogan reflects on the first 100 episodes of the Mathematica podcast, and retraces some of the steps taken in the almost five years since the series launched in early 2019.

Learn more, listen, and subscribe: https://mathematica.org/ontheevidence

Bonus Content | J.B. Wogan Reflects on 100 Episodes

Jennifer Pahlka served as the U.S. deputy chief technology officer under President Barack Obama and founded Code for America, a nonprofit that works to improve government digital services. In her new book, "Recoding America: Why Government is Failing in the Digital Age and How We Can Do Better," Pahlka takes us beyond the basics to explore why policy implementation goes wrong and what we can do to improve delivery of government services and create better outcomes for the American public.

On this episode of On the Evidence, Pahlka and Mathematica's Mike Burns discuss "Recoding America," the nexus of evidence-based policy and delivery-driven policy, and how we can close the gap between policy intentions and real-world outcomes.

A full transcript of this episode is available here: https://www.mathematica.org/blogs/jennifer-pahlka-on-government-in-the-digital-age

Learn more about Pahlka's book, "Recoding America: Why Government is Failing in the Digital Age and How We Can Do Better": https://us.macmillan.com/books/9781250266774/recodingamerica

Read Pahlka’s June 2023 op-ed in The Washington Post, “Better government tech starts with people. New Jersey shows how.”: https://www.washingtonpost.com/opinions/2023/06/13/new-jersey-digital-unemployment-insurance/

Read Pahlka’s bio: https://www.recodingamerica.us/

106 | Jennifer Pahlka on Government in the Digital Age

The latest episode of Mathematica’s On the Evidence podcast features Elizabeth Linos, the 23rd recipient of the David N. Kershaw Award and Prize. Linos is a public management scholar and behavioral scientist who studies, designs, and tests innovations in how government works. On the episode, Linos talks about what public sector workers and public policy researchers can learn from each other, how she selects research topics that have salience in the policy community, and what she is learning about scaling up and sustaining successful pilot projects in government.

A full transcript of the episode is available at mathematica.org/blogs/elizabeth-linos-on-using-evidence-to-bolster-the-public-sector-workforce-and-improve-government

107 | Elizabeth Linos on Bolstering the Public Sector Workforce and Improving Government Services

Early in the COVID-19 pandemic, inconsistent data collection and reporting made it difficult for U.S. public health agencies to respond to the disease's inequitable impacts. Demographic and socioeconomic factors, such as age, race, ethnicity, gender, income, and disability status, were particularly challenging to capture. The same data issues would later impede agencies’ ability to prioritize vaccinations for the people most impacted by the pandemic. Even though COVID-19 is no longer a global public health emergency, the underlying data problems remain. 
 
“What’s at stake is saving lives,” says Alonzo Plough, chief science officer and vice president of research, evaluation, and learning at the Robert Wood Johnson Foundation (RWJF), who joins us for this episode of Mathematica’s On the Evidence podcast. Plough explains, “bad data, lack of timely data, [and] lack of connected data” result in “missing opportunities for early intervention that can save lives.” 
 
In this two-part episode, Plough joins George Hobor, Javier Robles, and Anita Chandra, as they discuss the deficits of the U.S. public health data infrastructure, how these deficits affect health equity, and how public health agencies can improve their responses to public health crises by transforming their data systems.

- Hobor is a senior program officer at RWJF.
- Robles is director of the Center for Disability Sports, Health, and Wellness at Rutgers University and was a member of RWJF’s National Commission to Transform Public Health Data Systems.
- Chandra is vice president and director of RAND Social and Economic Well-Being at the RAND Corporation.

In part 1, Mathematica’s Deric Joyner speaks with Plough about the motivation behind the Transforming Public Health Data Systems initiative. 

In part 2, Mathematica’s Dave Roberts moderates a conversation between Hobor, Robles, and Chandra, about insights from the initiative and what changes need to happen next to improve the nation’s public health data infrastructure.

Part 1 is available here: https://on.soundcloud.com/iQcZ4

Transcripts for parts 1 and 2 are available here: https://mathematica.org/blogs/improving-health-equity-by-transforming-public-health-data-systems

109, Part 2 | Improving Health Equity by Transforming Public Health Data Systems

Early in the COVID-19 pandemic, inconsistent data collection and reporting made it difficult for U.S. public health agencies to respond to the disease's inequitable impacts. Demographic and socioeconomic factors, such as age, race, ethnicity, gender, income, and disability status, were particularly challenging to capture. The same data issues would later impede agencies’ ability to prioritize vaccinations for the people most impacted by the pandemic. Even though COVID-19 is no longer a global public health emergency, the underlying data problems remain. 
 
“What’s at stake is saving lives,” says Alonzo Plough, chief science officer and vice president of research, evaluation, and learning at the Robert Wood Johnson Foundation (RWJF), who joins us for this episode of Mathematica’s On the Evidence podcast. Plough explains, “bad data, lack of timely data, [and] lack of connected data” result in “missing opportunities for early intervention that can save lives.” 
 
In this two-part episode, Plough joins George Hobor, Javier Robles, and Anita Chandra, as they discuss the deficits of the U.S. public health data infrastructure, how these deficits affect health equity, and how public health agencies can improve their responses to public health crises by transforming their data systems.

- Hobor is a senior program officer at RWJF.
- Robles is director of the Center for Disability Sports, Health, and Wellness at Rutgers University and was a member of RWJF’s National Commission to Transform Public Health Data Systems.
- Chandra is vice president and director of RAND Social and Economic Well-Being at the RAND Corporation.

In part 1, Mathematica’s Deric Joyner speaks with Plough about the motivation behind the Transforming Public Health Data Systems initiative. 

In part 2, Mathematica’s Dave Roberts moderates a conversation between Hobor, Robles, and Chandra, about insights from the initiative and what changes need to happen next to improve the nation’s public health data infrastructure.

Part 2 is available here: https://on.soundcloud.com/ZoipS

Transcripts for parts 1 and 2 are available here: https://mathematica.org/blogs/improving-health-equity-by-transforming-public-health-data-systems

109, Part 1 | Improving Health Equity by Transforming Public Health Data Systems

Every year, early-career scholars join Mathematica for a summer to work on independent research related to improving public well-being. As prospective applicants prepare to apply by the Feb. 29 deadline for the 2024 Summer Fellowship program, Mathematica’s On the Evidence podcast spoke with last year’s fellows (Laurin Bixby, Carla Wellborn, and Luis Carlos Carvajal-Osorio) about what is at stake in the questions they’re investigating, why they’re interested in applied research, and what they found difficult and rewarding about applied public policy research. 

A full transcript is available at https://mathematica.org/blogs/using-schools-to-improve-child-health-why-black-families-choose-to-homeschool-their-kids-and-what

111 | Homeschooling, Inequities in Disability Policy, and Improving Child Health Through School

This episode of Mathematica’s On the Evidence podcast explores lessons from Maryland, with glimpses from Vermont and Pennsylvania, that could help other states decide whether and how to adopt hospital global budgeting. The guests for this episode are Dr. Joshua Sharfstein, who was the secretary of the Maryland Department of Health and Mental Hygiene when the state adopted hospital global budgeting; Thomas Mullen, who led a Baltimore-area hospital system when Maryland adopted the new payment model; and Dr. Sule Gerovich, a senior fellow at Mathematica who supported Vermont, Pennsylvania, and Maryland in developing hospital global budgets. 

Full transcript of the episode is available here: https://www.mathematica.org/blogs/what-states-can-learn-from-marylands-experience-with-hospital-global-budgeting

Subscribe for future episodes on your podcast player of choice here: mathematica.org/ontheevidence

112 | Lessons for States Interested in Hospital Global Budgeting

A new episode of Mathematica’s On the Evidence podcast explores the idea that a more comprehensive approach to evaluation, including study of a program’s design and implementation, maximizes a program’s chances of success. The episode draws from a convening in January at Mathematica’s D.C. office at which experts from the federal government, philanthropy, academia, and research organizations discussed insights from the Oxford Handbook on Program Design and Implementation Evaluation, edited by Mathematica’s Anu Rangarajan.

The episode features Rangarajan as well as seven other speakers who expand upon the idea that a more comprehensive approach to evaluation could provide more useful information about whether a program is working, and if not, how it could be tweaked to work. The following speakers also appear in the episode:

- Michael Woolcock, lead social scientist, Development Research Group, World Bank
- Paul Decker, president and chief executive officer, Mathematica
- Emilie Bagby, director, international education, Mathematica
- Alix Zwane, chief executive officer, Global Innovation Fund
- Melissa Chiappetta, senior education advisor, Latin America and the Caribbean, U.S. Agency for International Development
- Douglas J. Besharov, professor, University of Maryland School of Public Policy
- Michelle Sager, managing director for strategic issues, U.S. Government Accountability Office

A transcript of the episode is available at https://mathematica.org/blogs/improving-the-impact-of-social-programs-through-better-evaluation

Learn more about Mathematica Global, the new name and identity of Mathematica’s international unit: https://www.mathematica.org/blogs/mathematica-global-brings-hindsight-insight-and-foresight

Watch the full video recording from the January convening about improving the impact of social programs through a comprehensive approach to design and evaluation: https://www.mathematica.org/events/improving-the-impact-of-social-programs-through-a-comprehensive-approach-to-design-and-evaluation

Listen to a previous episode of Mathematica’s On the Evidence podcast that features two of the speakers from the January event, Emilie Bagby and Melissa Chiappetta, discussing how the U.S. Agency for International Development and the International Rescue Committee are building on an evidence review from Mathematica to help local education leaders implement effective programs and policies in northern Central America that will reduce local violence and crime: https://www.mathematica.org/blogs/developing-education-programs-to-prevent-violence-and-crime-in-latin-america-and-the-caribbean

115 | Improving the Impact of Social Programs Through Better Evaluation

The latest episode of On the Evidence features an interview with Mathematica’s Ngan MacDonald about the applications of artificial intelligence (AI) for improving health care through data analytics. MacDonald recently joined Mathematica as the company’s director of health data innovations, where she leads a team of data scientists that help public and private health organizations use their data to deliver meaningful and effective insights. In addition to her role at Mathematica, MacDonald is also the chief of data operations for the Institute for Artificial Intelligence in Medicine at Northwestern University. 

On the episode, MacDonald discusses the potential benefits of AI in health care, the equity implications of training AI on incomplete health care data, and what AI could mean for Mathematica’s work in data analytics. 

A full transcript of the interview is available at https://mathematica.org/blogs/improving-health-equity-through-ai-and-data-innovation

116 | Applying AI to Improve Health Policy Research

This episode of Mathematica’s On the Evidence podcast features a discussion about how various parties within the health care ecosystem can responsibly use artificial intelligence (AI) to improve patient health. The conversation comes amid increasing interest in the applications of AI in daily life and one month after Mathematica announced the launch of its Health Data Innovation Lab, a digital operational hub for government agencies, foundations, medical centers, and other health organizations to collaborate with data scientists and health policy experts. 

The episode draws from a recent webinar moderated by Noland Joiner, the chief technology officer of health care at Mathematica, with Ngan MacDonald and Ellie Graeden as guests. MacDonald is the director of health data innovations at Mathematica and Graeden is a partner and chief data scientist at Luminos.Law. She is also an adjunct research professor at Georgetown University.

A transcript of the episode is available here: https://www.mathematica.org/blogs/improving-health-through-responsible-ai

Learn more about Mathematica’s Health Data Innovation Lab: https://www.mathematica.org/sites/health-data-innovation-lab

120 | Improving Health through Responsible Use of AI

Within the past few years, organizations that help improve public well-being by providing data-driven insights have witnessed significant changes. From a digital transformation hastened by big data and artificial intelligence (AI) to the globalization of evidence-based solutions for problems that transcend borders, changes in technology, society, and culture are challenging leaders to rethink how their organizations operate. On the latest episode of Mathematica’s On the Evidence podcast, NORC at the University of Chicago President and Chief Executive Officer Dan Gaylin joined Mathematica President and Chief Executive Officer Paul Decker for a wide-ranging discussion about leadership, AI, globalization, the COVID-19 pandemic, and advocating for the use of high-quality evidence without straying into activism.

Find a full transcript at mathematica.org/blogs/norc-and-mathematica-ceos-on-the-future-of-data-and-evidence

Read an article by Dan Gaylin on LinkedIn where he explains his philosophy around the importance of sharing research findings: https://www.linkedin.com/pulse/from-silence-significance-case-sharing-your-research-dan-gaylin-d1cve/?trackingId=Mm6Fa4kN4WbH29J%2FQArQrg%3D%3D

Learn more about NORC’s AmeriSpeak Panel: https://amerispeak.norc.org/

Learn more about NORC’s 2020 Election Research Project for Facebook: https://www.norc.org/research/projects/2020-election-research-project.html

Learn more about NORC’s Advanced Data Solutions Center: https://www.norc.org/about/departments/advanced-data-solutions-center.html

Watch Decker’s interview with Mark Zandi, chief economist at Moody’s Analytics, which includes a discussion about the declining response rates in federal surveys and what they might mean for informing policy decisions: https://www.mathematica.org/blogs/mark-zandi-chief-economist-of-moodys-analytics-on-data-driven-decisions-in-public-policy

Watch Decker’s interview with Max Stier, the president and CEO of the Partnership for Public Service: https://www.mathematica.org/blogs/the-partnership-for-public-services-max-stier-on-using-data-to-improve-the-government

121 | NORC and Mathematica CEOs on the Future of Data and Evidence

In the same way a doctor uses data to diagnose and treat a patient, states’ public health agencies use data to measure and address health in their communities. In the United States, states have the autonomy to decide their own data policies, which influences the amount, quality, and timeliness of public health data they produce. On the heels of the worst global pandemic in a century—and at a time when technology is significantly changing the way that information flows to public health agencies—a new report from the Pew Charitable Trusts offers a first-of-its-kind glimpse into the patchwork of state data policies and practices that together form our modern system for the nation’s public health data. 

The latest episode of Mathematica’s On the Evidence podcast features a discussion about the report’s implications with Lilly Kan of the Centers for Disease Control and Prevention, Annie Fine of the Council of State and Territorial Epidemiologists, Mary Beth Kurilo of the American Immunization Registry Association, and Monica Huang of Mathematica. 

A full transcript of the episode is available at mathematica.org/blogs/modernizing-state-public-health-data-policies-and-practices

Read a blog by The Pew Charitable Trusts' Margaret Arneson and Kathy Talkington about key takeaways from the report: https://www.pewtrusts.org/en/research-and-analysis/articles/2024/12/12/states-must-modernize-public-health-data-reporting-new-report-finds-promising-practices

Read the full report from the Pew Charitable Trusts, which was supported with research by Mathematica: https://www.pewtrusts.org/en/research-and-analysis/reports/2024/12/state-public-health-data-reporting-policies-and-practices-vary-widely

Read a LinkedIn article from Kathy Talkington of The Pew Charitable Trusts discussing key takeaways from the report: https://www.linkedin.com/pulse/public-health-data-reporting-strengths-shortcomings-kathy-talkington-gcdne/?trackingId=qNW27JNuSXq2Uf67iWKD9g%3D%3D

Take an online quiz to test how much you know about public health data reporting: https://www.pewtrusts.org/en/research-and-analysis/articles/2024/12/12/how-much-do-you-know-about-public-health-data-reporting

134 | Modernizing State Public Health Data Policies and Practices

Cecilia Rouse is the former dean of the Princeton School of Public and International Affairs and the current president of the Brookings Institution. For the first two years of the Biden administration, she was the chair of the Council of Economic Advisers—the first Black American to chair the council in its 75-year history. In August, she joined Mathematica’s On the Evidence podcast for a conversation with Mathematica’s President and Chief Executive Officer Paul Decker about leadership, the use of evidence in public policy, and the role of research organizations in improving public well-being.

During the interview, Rouse talks about what she learned from her government posts about the role of research in informing policy decisions, how she thinks about the role of academic institutions in conducting policy research and training future public servants, how economics and related professions are doing at diversifying their workforces, and what she has learned about leadership.

A full transcript of the episode is available at https://mathematica.org/blogs/cecilia-rouse-on-the-use-of-evidence-in-public-policy

Listen to Rouse’s January 2024 interview with the Brookings Institution’s podcast, The Current, which posted shortly after she became the Brookings president: https://www.brookings.edu/articles/meet-cecilia-rouse-new-brookings-president/

Listen to Paul Decker’s previous interviews about leadership and evidence-based decision making: https://staginginter.mathematica.net/search#q=paul%20decker&sort=relevancy&f:BlogSeries=%5BOn%20The%20Evidence%2DPodcast%5D&f:BlogSeries:operator=and

127 | Ensuring Evidence Use in Public Policy with Brookings Institution Pres. Cecilia Rouse

A new episode of Mathematica’s On the Evidence podcast explores recent research that experimented with a new approach to equitable data collection that also equips individuals and their organizations with new skills and resources. The Robert Wood Johnson Foundation contracted with Mathematica and its partner UBUNTU Research and Evaluation to learn how grassroots organizations in the housing justice movement had used grant funds for community power building. 

Through a fellowship program established by Mathematica and UBUNTU, researchers provided training in evaluation to fellows selected from staff at grassroots organizations while learning from those staff about the impact of the foundation’s investment. The research also provided early evidence that the foundation’s investment helped local grassroots organizations become more effective in achieving their affordable housing goals. On the episode, researchers and program fellows discuss the impact of the foundation’s investment and how the fellowship model helped strengthen organizations’ internal capacity to learn from campaigns and make evidence-informed changes.

Find a full transcript of the episode at mathematica.org/blogs/how-evaluation-can-support-housing-justice-and-community-change

Learn more about the housing justice and community power building evaluation conducted by Mathematica and UBUNTU on behalf of the Robert Wood Johnson Foundation: https://www.ubunturesearch.com/hjet

Explore the interactive graphic that illustrates a new framework for community power building in the housing justice movement: https://mathematica.org/features/the-house-that-justice-built
 
Learn more about the recent (open source) book Drew Koleros edited on updating theories of change for the field of evaluation, which includes sections that are relevant for evaluating community power building: https://www.routledge.com/Theories-of-Change-in-Reality-Strengths-Limitations-and-Future-Directions/Koleros-Adrien-Tyrrell/p/book/9781032669588?srsltid=AfmBOooKlrBZhZa5lb2HnU1K2Nw-IjWyQQ660RW0qfWxzw9P3FUTYpMU

129 | How Evaluation Can Support Housing Justice and Community Change

The latest episode of Mathematica’s On the Evidence podcast features Jeffery Marino, the director of California’s Office of Data and Innovation (ODI). For the podcast, Mathematica’s President and Chief Executive Officer Paul Decker spoke with Marino about ODI’s CalAcademy for training state employees in areas such as plain language and human-centered design, California Governor Gavin Newsom’s executive order on generative artificial intelligence, facilitating interagency data sharing agreements, and data-driven storytelling about government’s impact in people’s lives. 

They also discussed Marino’s career trajectory: he majored in English literature in college, became a data journalist, and now leads ODI, a department within California’s Government Operations Agency encouraging collaboration across state agencies to improve digital services for all Californians.

Find the full transcript of the interview: https://www.mathematica.org/blogs/embedding-data-and-innovation-across-california-state-government

Learn more about ODI’s CalAcademy: https://innovation.ca.gov/who-we-are/calacademy/

Read California Governor Newsom’s executive order on generative AI: https://www.gov.ca.gov/wp-content/uploads/2023/09/AI-EO-No.12-_-GGN-Signed.pdf
 
Listen to an On the Evidence interview with Jennifer Pahlka, author of Recoding America: Why Government is Failing in the Digital Age and How We Can Do Better, which is referenced in the discussion between Decker and Marino: https://mathematica.org/blogs/jennifer-pahlka-on-government-in-the-digital-age

Explore a playlist of interviews between Mathematica’s Paul Decker and fellow leaders of evidence-driven organizations: https://soundcloud.com/ontheevidence/sets/evidence-leadership?si=ea534d2c041c44e99c289a8d4dffa664&utm_source=clipboard&utm_medium=text&utm_campaign=social_sharing

132 | Embedding Data and Innovation Across California State Government

In honor of Employee Ownership Month in October, Mathematica’s On the Evidence podcast features a special episode with Paul Decker, Mathematica’s president and chief executive officer, and Jim Bonham, the president and chief executive officer of The ESOP Association. On the episode, Decker and Bonham discuss the Employee Stock Ownership Plan (ESOP) as an evidence-based tool that benefits employees, consumers, and communities across the nation.
 
An ESOP is a retirement vehicle for employees that many companies use to increase employee engagement and retain talent over time. As Decker and Bonham discuss on the episode, an ESOP can be more than a nice perk companies offer their employees. Business leaders and policymakers across the political spectrum have applauded the potential of ESOPs to strengthen the middle class and keep jobs in local communities. Research even shows ESOPs help reduce gender and racial wealth gaps.

On the episode, Decker and Bonham discuss the benefits of ESOPs and challenges facing ESOPs going forward. 

A full transcript of the episode is available at https://www.mathematica.org/blogs/supporting-workers-and-strengthening-communities-through-employee-ownership

Read the study from Institute for the Study of Employee Ownership and Profit Sharing within the School of Management and Labor Relations at Rutgers University that found ESOPs help narrow gender and racial wealth gaps: https://smlr.rutgers.edu/sites/smlr/files/Documents/News/rutgerskelloggreport_april2019.pdf

Read an article in The Star-Ledger about the benefits of employee stock ownership plans for reducing wealth inequality and ensuring the longevity of a business, which describes Mathematica’s experience with becoming an ESOP and quotes Paul Decker: https://www.nj.com/opinion/2024/09/nj-business-owners-need-a-succession-plan-and-workers-need-savings-we-can-solve-both-problems-opinion.html

Learn more about Mathematica’s history as an employee-owned company: https://mathematica.org/blogs/employee-ownership-is-at-the-heart-of-mathematica

130 | The Evidence-Based Benefits of Employee Ownership

The latest episode of Mathematica’s On the Evidence podcast features an interview with Katherine Michelmore, an associate professor of public policy at the University of Michigan’s Gerald R. Ford School of Public Policy and the 24th recipient of the David N. Kershaw Award and Prize. 

Michelmore’s research has focused on temporary expansions of tax credits during the COVID-19 pandemic, which is part of a growing body of evidence informing state and federal policy proposals to make permanent some or all of those changes. 

On the episode, Michelmore talks about her experiences interacting with the media and policymakers about the subject of her research, using a novel source of data from a private mobile app to study the impacts of an expanded Child Tax Credit on households, and questions she would like to pursue in the future related to tax credits that support working parents and their children. 

Find the full transcript at mathematica.org/blogs/kershaw-award-winner-katherine-michelmore-on-how-expanded-tax-credits-benefited-family-well-being

Check out the Spotify playlist with interviews with the 2020, 2021, 2022, and 2023 winners of the Kershaw Award: https://open.spotify.com/playlist/1Qz9HKUOxBhj33neIpPEUd?si=9oBu3VV2QJig8nUgKVVUKg&nd=1&dlsi=c61b4fd6357f426d

Going to the 2024 APPAM Fall Research Conference? On the Evidence will be there, too. Find us at the Mathematica booth. Send us a message at jwogan@mathematica-mpr.com to let us know you're coming. 

Read a working paper co-authored by Michelmore on the effects of the temporary expansion of the CTC on the economic well-being of families, including a reduction in food insecurity: https://www.nber.org/papers/w30533

Read an article co-authored by Michelmore on the effects of the temporary expansion of the CTC on housing affordability and the living arrangements of families: https://read.dukeupress.edu/demography/article/61/4/1069/389197/The-Effects-of-the-2021-Child-Tax-Credit-on

Read an article co-authored by Michelmore on the effects of the temporary expansion of the CTC on short- and long-term child development, including the likely improvement of children’s health: https://journals.sagepub.com/doi/10.1177/00027162241264412

Read a working paper co-authored by Michelmore on the effects of the temporary expansion of the childless EITC: https://www.nber.org/papers/w32571

131 | How Expanded Tax Credits Benefited Family Well-Being

On the latest episode of Mathematica’s On the Evidence podcast, Lauren Supplee, the deputy assistant secretary for planning, research, and evaluation at the Administration for Children and Families, sat down for a wide-ranging conversation about the use of evidence in social policy. Supplee reflected on this year’s White House blueprint on social and behavioral science in government, which included Head Start as an example of a government program for children and families that benefited from decades of social policy research overseen by the Administration for Children and Families. On the episode, Supplee also discussed the role of research in informing improvements to Head Start over time, how the Foundations for Evidence-Based Policymaking Act is changing the use of evidence in the federal government, and insights from Supplee’s blog series on boosting the use of research evidence.

Find a full transcript of the conversation here: https://mathematica.org/blogs/opres-lauren-supplee-on-boosting-the-use-of-social-policy-research

Read the White House Office of Science and Technology blog announcing its Blueprint for the Use of Social and Behavioral Science to Advance Evidence-Based Policymaking: https://www.whitehouse.gov/ostp/news-updates/2024/05/15/applying-social-and-behavioral-science-to-federal-policies-and-programs-to-deliver-better-outcomes/

Read Lauren Supplee’s blog about measuring whether and how evidence is used: https://www.acf.hhs.gov/opre/blog/2024/01/how-do-we-measure-whether-and-how-evidence-used

Read Supplee’s blog about cultivating more “knowledge brokers” in social policy research who translate complex data into action-ready insights: https://www.acf.hhs.gov/opre/blog/2024/07/can-we-all-develop-superpowers-knowledge-brokers

Read Supplee’s blog about establishing systems to support the use of evidence: https://www.acf.hhs.gov/opre/blog/2024/04/evidence-action-how-do-we-establish-systems-support-evidence-use

Read a blog by Colleen Rathgeb, the former director of policy at the Office of Head Start and current associate deputy assistant secretary for the Office of Early Childhood Development, about research showing the need for full-day, year-round Head Start programs: https://www.acf.hhs.gov/archive/blog/2015/02/presidents-fy-2016-budget-head-start

Explore the ELOF 2 Go mobile app, the free online tool Supplee references that supports teachers who want to access and learn more about the Head Start Early Learning Outcomes Framework (ELOF): https://play.google.com/store/apps/details?id=com.ohs.elof2go&hl=en_US&pli=1

Watch the video series referenced by Supplee that shares the perspectives and experiences of those who are involved in obtaining and using data from the American Indian and Alaska Native Head Start Family and Child Experiences Survey: https://www.acf.hhs.gov/opre/training-technical-assistance/overview-aian-faces-video-series

133 | Boosting the Use of Social Policy Research

This episode features audio from an in-person roundtable discussion at the 2024 Association for Public Policy Analysis and Management Fall Research Conference. The roundtable focused on the five-year anniversary of the Foundations for Evidence-Based Policymaking Act (Evidence Act) and a recent report from the Government Accountability Office (GAO) on federal agencies' ability to assess their capacity to collect and use evidence. The panelists also discussed what needs to happen next to further strengthen the use of evidence in the federal government. 

The roundtable featured remarks from Erika Rissi, Nick Hart, Melinda Buntin, Rachel Snyderman, and Paul Decker. 

Rissi is the chief evaluation officer of the National Science Foundation, where she is also the Head of its Evaluation and Assessment Capability Section.

Hart is the president and CEO of the Data Foundation. 

Buntin is a professor at the Johns Hopkins Bloomberg School of Public Health and the Carey Business School. 

Snyderman is the managing director of the Bipartisan Policy Center’s Economic Policy Program. 

Decker is the president and chief executive officer of Mathematica. 

You can also read a short blog by Decker summarizing key themes from the roundtable here: https://mathematica.org/blogs/advancing-and-enhancing-evidence-based-policymaking

Listen to a past episode of On the Evidence featuring a discussion about the five-year anniversary of the Evidence Act with experts from government, think tanks, research organizations, and academia: https://mathematica.org/blogs/building-on-the-evidence-act-to-increase-the-positive-impact-of-federal-policymaking

Listen to a past episode of On the Evidence featuring Robert Shea, an expert on performance improvement in government who served on the bipartisan U.S. Commission on Evidence-Based Policymaking, a group whose recommendations informed the Evidence Act: https://mathematica.org/blogs/robert-shea-on-the-federal-governments-progress-in-using-evidence-to-improve-programs-and-policies

135 | It’s the Evidence, Stupid: GAO’s Report on Evidence-Based Policymaking and What Comes Next

On the latest episode of Mathematica’s On the Evidence podcast, M-KOPA Chief Executive Officer Jesse Moore talks about increasing access to financial and digital services for unbanked workers in Africa by leasing out smartphones. He and Paul Decker, the president and chief executive officer of Mathematica, discuss the needs of everyday earners in Africa who work regular but informal jobs that make it difficult to build credit and open a bank account. They also discuss the role of data in informing an organization’s business strategy as well as assessing its social impact on customers. 

Find a blog summarizing highlights from the conversation, plus the full transcript here: mathematica.org/blogs/how-a-fintech-company-helps-workers-in-africa-access-credit-and-savings-with-smartphones

136 | How a Fintech Company Helps Workers in Africa Access Credit and Savings with Smartphones

On the latest episode of Mathematica’s On the Evidence podcast, Dr. Agnes Kalibata reflects on her career as a scientist and as the former president of the Alliance for a Green Revolution in Africa (AGRA), an African-led organization that seeks to create an environment where Africa can sustainably feed itself. It does so by focusing on scaling agricultural innovations that help smallholder farmers achieve increased incomes, better livelihoods, and improved food security.

Kalibata’s 10-year term as AGRA president ended earlier this year. She spoke with Mathematica President and Chief Executive Officer Paul Decker last December. Mathematica supports AGRA’s implementation of its 2023–2027 monitoring, evaluation, and learning strategy. On the episode, Kalibata and Decker discuss locally led development and the role of data in helping to understand how a program, such as AGRA’s Seed Systems, can be more effective.

Find a full transcript of the conversation here: https://mathematica.org/blogs/increasing-the-resilience-of-african-smallholder-farmers

Learn more about Mathematica’s work supporting AGRA as AGRA implements its 2023–2027 monitoring, evaluation, and learning strategy: https://mathematica.org/news/measuring-agras-impact-transforming-agricultural-systems-and-improving-climate-resilience

137 | Increasing the Resilience of African Smallholder Farmers

Rising healthcare costs in the U.S. threaten people’s access to treatment and services while reducing their ability to afford other necessities. Over the last decade, state policymakers across the political spectrum have responded to this urgent problem by passing legislation, setting up new government offices, and adding regulations to control healthcare spending. 


One critical element of states’ maturing strategies for addressing healthcare costs is high quality, timely, and accessible data. 


In the latest episode of Mathematica’s On the Evidence podcast, Jim Lloyd of the New Jersey Department of Health, Rachel Block of the Milbank Memorial Fund, and Julie Sonier of Mathematica discuss why rising healthcare costs present a complex and urgent issue, how states are responding, and the role of data in supporting solutions that address healthcare cost growth. 


“We have this healthcare system with much higher costs than other countries around the world, but we also have less access,” Lloyd explains. “There's an opportunity to be able to identify those costs that are contributing to quality, identify those costs that aren't, and then increase access and increase quality, potentially without increasing costs.” 


A blog summarizing the episode, with quotes from the guests and additional resources for further learning are available at https://mathematica.org/blogs/turning-data-into-solutions-for-reducing-healthcare-cost-growth-in-states

138 | Addressing Rising Healthcare Costs in States

Earlier this year, the Centers for Medicare & Medicaid Services (CMS) announced that it would invite medical practices to voluntarily participate in new payment models for primary care in the Medicare program. The initiative is called Primary Cares, and its goals are to reduce Medicare spending and improve both the quality of and access to primary care for Medicare beneficiaries—especially for those with complex, chronic conditions and serious illness. CMS officials believe that the new payment models could affect as many as 11 million Medicare fee-for-service beneficiaries and an estimated one in four primary care practitioners.

In light of that announcement, I invited three primary care experts from Mathematica to explain the latest push by CMS to change primary care through new payment models. My guests are Jeffrey Ballou, Eugene Rich, and Julie Schilz, who have the combined experience and expertise of a health economist, a general internist, a medical practice leader, a registered nurse, and a former senior executive at a national health insurance company.

We also have a condensed Q&A version of the conversation available here: https://www.mathematica-mpr.com/commentary/federal-efforts-to-overhaul-the-way-we-pay-for-primary-care-explained

10 | Federal Efforts to Overhaul the Way We Pay for Primary Care, Explained

Our economy has always had jobs that fit outside the conventional 9 to 5, Monday to Friday, schedule, but with the rise of telework, on-demand scheduling apps, "irregular work," and the Gig Economy, more workers not only find themselves earning their paychecks in the early morning, at night or on weekends; but their hours may change on a frequent basis.  

Finding safe and high quality child care that accommodates non-standard or changing work schedules is a familiar challenge for many workers, but a relatively new area of public policy research. 

On this week's episode of On the Evidence, we talk about a report that looks at the complicated relationships between changing work schedules, the availability of child care for those schedules, and child well-being.

My guests are the co-authors of that report, Angela Rachidi and Rus Sykes, who conducted research on behalf of Mathematica and the American Public Human Services Association, with support from the Robert Wood Johnson Foundation.  

An abridged and edited Q&A blog based on the conversation is available here: https://mathematica-mpr.com/commentary/the-complex-relationship-between-changing-work-schedules-child-care-and-child-well-being

15 | The Relationship Between Changing Work Schedules, Child Care, and Child Well-Being

A podcast from Mathematica that examines what we know about today’s most urgent challenges and how we can make progress in addressing them. Reimagining the way the world gathers and uses data, Mathematica uncovers the evidence that offers our partners the confidence and clarity they need to find out what can be done, how to make it happen, and where to go next.

On the Evidence

Pinned to spotlight

Popular tracks