In May 2023, Mathematica hosted a convening on Capitol Hill about embedding evidence in federal decision making, with a focus on the legacy of the nearly five-year-old Foundations for Evidence-Based Policymaking Act, also known as the Evidence Act. One of the attendees that day was Robert Shea, an expert on performance improvement in government who served on the bipartisan U.S. Commission on Evidence-Based Policymaking, a group whose recommendations informed the Evidence Act. Shea is the chief executive officer for GovNavigators, a government management consulting firm, where he hosts a podcast called The GovNavigators Show. His career has also included posts at the White House Office of Management and Budget, the Senate Committee on Governmental Affairs, and the House Committee on Government Reform and Oversight.

On this episode of Mathematica’s On the Evidence podcast, Shea discusses the improvements he has witnessed since the mid-1990s in the use of data and evidence in federal decision making.

“We have seen a sea change in the ability of agencies to understand that they need to articulate what they’re trying to accomplish in terms of outcomes, report that transparently,” Shea says on the episode. “A lot of what we’re talking about today is sort of dig deeper—find out whether it’s what we’re doing that’s contributing to the ultimate outcome rather than some other factor.”

This episode is part of an occasional series on the show called Evidence in Government, which explores new developments in the halls of government and the role that evidence can play in decisions that could improve people’s lives. Mike Burns, Mathematica’s senior director of communications and public affairs at Mathematica, conducts the Evidence in Government interviews.

SHOW NOTES

Listen to The GovNavigators Show, a podcast Shea hosts with Adam Hughes about government management: https://www.govnavigators.com/podcast

Watch a recording of the event Mathematica hosted in May 2023 on the Evidence Act: https://www.mathematica.org/events/evidence-at-work-for-the-american-people

Learn more about the U.S. Commission on Evidence-Based Policymaking, which made recommendations ahead of the Foundations for Evidence-Based Policymaking Act of 2018: https://obamawhitehouse.archives.gov/omb/management/commission_evidence

Read a fact sheet from the Bipartisan Policy Center on the Foundations for Evidence-Based Policymaking Act: https://bipartisanpolicy.org/blog/fact-sheet-foundations-for-evidence-based-policymaking-act/

Read a press release about the resolution to establish a new commission on evidence-based policymaking: https://kilmer.house.gov/media/press-releases/kilmer-spearheads-legislative-change-with-introduction-of-evidence-based-policymaking-resolution

Read the bipartisan resolution to establish a new commission on evidence-based policymaking to further embed the use of data and evidence in federal decision making: https://www.congress.gov/bill/118th-congress/house-concurrent-resolution/49/text?s=1&r=1&q=%7B%22search%22%3A%5B%22Evidence+commission%22%5D%7D

100 | Robert Shea on the Federal Government’s Progress in Using Evidence

In the same way a doctor uses data to diagnose and treat a patient, states’ public health agencies use data to measure and address health in their communities. In the United States, states have the autonomy to decide their own data policies, which influences the amount, quality, and timeliness of public health data they produce. On the heels of the worst global pandemic in a century—and at a time when technology is significantly changing the way that information flows to public health agencies—a new report from the Pew Charitable Trusts offers a first-of-its-kind glimpse into the patchwork of state data policies and practices that together form our modern system for the nation’s public health data. 

The latest episode of Mathematica’s On the Evidence podcast features a discussion about the report’s implications with Lilly Kan of the Centers for Disease Control and Prevention, Annie Fine of the Council of State and Territorial Epidemiologists, Mary Beth Kurilo of the American Immunization Registry Association, and Monica Huang of Mathematica. 

A full transcript of the episode is available at mathematica.org/blogs/modernizing-state-public-health-data-policies-and-practices

Read a blog by The Pew Charitable Trusts' Margaret Arneson and Kathy Talkington about key takeaways from the report: https://www.pewtrusts.org/en/research-and-analysis/articles/2024/12/12/states-must-modernize-public-health-data-reporting-new-report-finds-promising-practices

Read the full report from the Pew Charitable Trusts, which was supported with research by Mathematica: https://www.pewtrusts.org/en/research-and-analysis/reports/2024/12/state-public-health-data-reporting-policies-and-practices-vary-widely

Read a LinkedIn article from Kathy Talkington of The Pew Charitable Trusts discussing key takeaways from the report: https://www.linkedin.com/pulse/public-health-data-reporting-strengths-shortcomings-kathy-talkington-gcdne/?trackingId=qNW27JNuSXq2Uf67iWKD9g%3D%3D

Take an online quiz to test how much you know about public health data reporting: https://www.pewtrusts.org/en/research-and-analysis/articles/2024/12/12/how-much-do-you-know-about-public-health-data-reporting

134 | Modernizing State Public Health Data Policies and Practices

The latest episode of Mathematica’s On the Evidence podcast focuses on the potential for health data interoperability to improve people’s health and well-being. Improved data interoperability is part of a broader push in the public and private sectors to use digital technology to make greater volumes of data available faster, at lower cost, and in higher-quality formats. These advances would make data easier to access, especially when needed to prevent or address urgent problems. In health care, the digital transformation in data could keep people healthier by improving the speed and quality of care patients receive. 

The episode features a discussion that originally aired as a webinar hosted by Mathematica’s Health Data Innovation Lab, which examined strategies and tools for achieving greater data interoperability. During the webinar, Mathematica’s Steve Linthicum moderated a conversation between Dr. Abel Kho of the Institute for Artificial Intelligence in Medicine at Northwestern University, Steven Gruner of HealthWare Systems, and Nicholai Mitchko of InterSystems. 

A full transcript of the episode is available at https://www.mathematica.org/blogs/how-health-data-interoperability-can-improve-patient-care

Learn more about Mathematica’s Health Data Innovation Lab: https://www.mathematica.org/sites/health-data-innovation-lab

Watch the previous webinar hosted by Mathematica’s Health Data Innovation Lab on data governance: https://www.mathematica.org/blogs/event-takeaways-using-data-governance-to-improve-health-outcomes

Watch the previous webinar hosted by Mathematica’s Health Data Innovation Lab on artificial intelligence: https://www.mathematica.org/news/health-ai-event-recap-its-all-about-the-data

128 | How Better Data Interoperability Can Improve Care Delivered To Patients

A new episode of Mathematica’s On the Evidence podcast explores recent research that experimented with a new approach to equitable data collection that also equips individuals and their organizations with new skills and resources. The Robert Wood Johnson Foundation contracted with Mathematica and its partner UBUNTU Research and Evaluation to learn how grassroots organizations in the housing justice movement had used grant funds for community power building. 

Through a fellowship program established by Mathematica and UBUNTU, researchers provided training in evaluation to fellows selected from staff at grassroots organizations while learning from those staff about the impact of the foundation’s investment. The research also provided early evidence that the foundation’s investment helped local grassroots organizations become more effective in achieving their affordable housing goals. On the episode, researchers and program fellows discuss the impact of the foundation’s investment and how the fellowship model helped strengthen organizations’ internal capacity to learn from campaigns and make evidence-informed changes.

Find a full transcript of the episode at mathematica.org/blogs/how-evaluation-can-support-housing-justice-and-community-change

Learn more about the housing justice and community power building evaluation conducted by Mathematica and UBUNTU on behalf of the Robert Wood Johnson Foundation: https://www.ubunturesearch.com/hjet

Explore the interactive graphic that illustrates a new framework for community power building in the housing justice movement: https://mathematica.org/features/the-house-that-justice-built
 
Learn more about the recent (open source) book Drew Koleros edited on updating theories of change for the field of evaluation, which includes sections that are relevant for evaluating community power building: https://www.routledge.com/Theories-of-Change-in-Reality-Strengths-Limitations-and-Future-Directions/Koleros-Adrien-Tyrrell/p/book/9781032669588?srsltid=AfmBOooKlrBZhZa5lb2HnU1K2Nw-IjWyQQ660RW0qfWxzw9P3FUTYpMU

129 | How Evaluation Can Support Housing Justice and Community Change

The latest episode of Mathematica’s On the Evidence podcast features Jeffery Marino, the director of California’s Office of Data and Innovation (ODI). For the podcast, Mathematica’s President and Chief Executive Officer Paul Decker spoke with Marino about ODI’s CalAcademy for training state employees in areas such as plain language and human-centered design, California Governor Gavin Newsom’s executive order on generative artificial intelligence, facilitating interagency data sharing agreements, and data-driven storytelling about government’s impact in people’s lives. 

They also discussed Marino’s career trajectory: he majored in English literature in college, became a data journalist, and now leads ODI, a department within California’s Government Operations Agency encouraging collaboration across state agencies to improve digital services for all Californians.

Find the full transcript of the interview: https://www.mathematica.org/blogs/embedding-data-and-innovation-across-california-state-government

Learn more about ODI’s CalAcademy: https://innovation.ca.gov/who-we-are/calacademy/

Read California Governor Newsom’s executive order on generative AI: https://www.gov.ca.gov/wp-content/uploads/2023/09/AI-EO-No.12-_-GGN-Signed.pdf
 
Listen to an On the Evidence interview with Jennifer Pahlka, author of Recoding America: Why Government is Failing in the Digital Age and How We Can Do Better, which is referenced in the discussion between Decker and Marino: https://mathematica.org/blogs/jennifer-pahlka-on-government-in-the-digital-age

Explore a playlist of interviews between Mathematica’s Paul Decker and fellow leaders of evidence-driven organizations: https://soundcloud.com/ontheevidence/sets/evidence-leadership?si=ea534d2c041c44e99c289a8d4dffa664&utm_source=clipboard&utm_medium=text&utm_campaign=social_sharing

132 | Embedding Data and Innovation Across California State Government

In honor of Employee Ownership Month in October, Mathematica’s On the Evidence podcast features a special episode with Paul Decker, Mathematica’s president and chief executive officer, and Jim Bonham, the president and chief executive officer of The ESOP Association. On the episode, Decker and Bonham discuss the Employee Stock Ownership Plan (ESOP) as an evidence-based tool that benefits employees, consumers, and communities across the nation.
 
An ESOP is a retirement vehicle for employees that many companies use to increase employee engagement and retain talent over time. As Decker and Bonham discuss on the episode, an ESOP can be more than a nice perk companies offer their employees. Business leaders and policymakers across the political spectrum have applauded the potential of ESOPs to strengthen the middle class and keep jobs in local communities. Research even shows ESOPs help reduce gender and racial wealth gaps.

On the episode, Decker and Bonham discuss the benefits of ESOPs and challenges facing ESOPs going forward. 

A full transcript of the episode is available at https://www.mathematica.org/blogs/supporting-workers-and-strengthening-communities-through-employee-ownership

Read the study from Institute for the Study of Employee Ownership and Profit Sharing within the School of Management and Labor Relations at Rutgers University that found ESOPs help narrow gender and racial wealth gaps: https://smlr.rutgers.edu/sites/smlr/files/Documents/News/rutgerskelloggreport_april2019.pdf

Read an article in The Star-Ledger about the benefits of employee stock ownership plans for reducing wealth inequality and ensuring the longevity of a business, which describes Mathematica’s experience with becoming an ESOP and quotes Paul Decker: https://www.nj.com/opinion/2024/09/nj-business-owners-need-a-succession-plan-and-workers-need-savings-we-can-solve-both-problems-opinion.html

Learn more about Mathematica’s history as an employee-owned company: https://mathematica.org/blogs/employee-ownership-is-at-the-heart-of-mathematica

130 | The Evidence-Based Benefits of Employee Ownership

The latest episode of Mathematica’s On the Evidence podcast features an interview with Katherine Michelmore, an associate professor of public policy at the University of Michigan’s Gerald R. Ford School of Public Policy and the 24th recipient of the David N. Kershaw Award and Prize. 

Michelmore’s research has focused on temporary expansions of tax credits during the COVID-19 pandemic, which is part of a growing body of evidence informing state and federal policy proposals to make permanent some or all of those changes. 

On the episode, Michelmore talks about her experiences interacting with the media and policymakers about the subject of her research, using a novel source of data from a private mobile app to study the impacts of an expanded Child Tax Credit on households, and questions she would like to pursue in the future related to tax credits that support working parents and their children. 

Find the full transcript at mathematica.org/blogs/kershaw-award-winner-katherine-michelmore-on-how-expanded-tax-credits-benefited-family-well-being

Check out the Spotify playlist with interviews with the 2020, 2021, 2022, and 2023 winners of the Kershaw Award: https://open.spotify.com/playlist/1Qz9HKUOxBhj33neIpPEUd?si=9oBu3VV2QJig8nUgKVVUKg&nd=1&dlsi=c61b4fd6357f426d

Going to the 2024 APPAM Fall Research Conference? On the Evidence will be there, too. Find us at the Mathematica booth. Send us a message at jwogan@mathematica-mpr.com to let us know you're coming. 

Read a working paper co-authored by Michelmore on the effects of the temporary expansion of the CTC on the economic well-being of families, including a reduction in food insecurity: https://www.nber.org/papers/w30533

Read an article co-authored by Michelmore on the effects of the temporary expansion of the CTC on housing affordability and the living arrangements of families: https://read.dukeupress.edu/demography/article/61/4/1069/389197/The-Effects-of-the-2021-Child-Tax-Credit-on

Read an article co-authored by Michelmore on the effects of the temporary expansion of the CTC on short- and long-term child development, including the likely improvement of children’s health: https://journals.sagepub.com/doi/10.1177/00027162241264412

Read a working paper co-authored by Michelmore on the effects of the temporary expansion of the childless EITC: https://www.nber.org/papers/w32571

131 | How Expanded Tax Credits Benefited Family Well-Being

On the latest episode of Mathematica’s On the Evidence podcast, Lauren Supplee, the deputy assistant secretary for planning, research, and evaluation at the Administration for Children and Families, sat down for a wide-ranging conversation about the use of evidence in social policy. Supplee reflected on this year’s White House blueprint on social and behavioral science in government, which included Head Start as an example of a government program for children and families that benefited from decades of social policy research overseen by the Administration for Children and Families. On the episode, Supplee also discussed the role of research in informing improvements to Head Start over time, how the Foundations for Evidence-Based Policymaking Act is changing the use of evidence in the federal government, and insights from Supplee’s blog series on boosting the use of research evidence.

Find a full transcript of the conversation here: https://mathematica.org/blogs/opres-lauren-supplee-on-boosting-the-use-of-social-policy-research

Read the White House Office of Science and Technology blog announcing its Blueprint for the Use of Social and Behavioral Science to Advance Evidence-Based Policymaking: https://www.whitehouse.gov/ostp/news-updates/2024/05/15/applying-social-and-behavioral-science-to-federal-policies-and-programs-to-deliver-better-outcomes/

Read Lauren Supplee’s blog about measuring whether and how evidence is used: https://www.acf.hhs.gov/opre/blog/2024/01/how-do-we-measure-whether-and-how-evidence-used

Read Supplee’s blog about cultivating more “knowledge brokers” in social policy research who translate complex data into action-ready insights: https://www.acf.hhs.gov/opre/blog/2024/07/can-we-all-develop-superpowers-knowledge-brokers

Read Supplee’s blog about establishing systems to support the use of evidence: https://www.acf.hhs.gov/opre/blog/2024/04/evidence-action-how-do-we-establish-systems-support-evidence-use

Read a blog by Colleen Rathgeb, the former director of policy at the Office of Head Start and current associate deputy assistant secretary for the Office of Early Childhood Development, about research showing the need for full-day, year-round Head Start programs: https://www.acf.hhs.gov/archive/blog/2015/02/presidents-fy-2016-budget-head-start

Explore the ELOF 2 Go mobile app, the free online tool Supplee references that supports teachers who want to access and learn more about the Head Start Early Learning Outcomes Framework (ELOF): https://play.google.com/store/apps/details?id=com.ohs.elof2go&hl=en_US&pli=1

Watch the video series referenced by Supplee that shares the perspectives and experiences of those who are involved in obtaining and using data from the American Indian and Alaska Native Head Start Family and Child Experiences Survey: https://www.acf.hhs.gov/opre/training-technical-assistance/overview-aian-faces-video-series

133 | Boosting the Use of Social Policy Research

This episode features audio from an in-person roundtable discussion at the 2024 Association for Public Policy Analysis and Management Fall Research Conference. The roundtable focused on the five-year anniversary of the Foundations for Evidence-Based Policymaking Act (Evidence Act) and a recent report from the Government Accountability Office (GAO) on federal agencies' ability to assess their capacity to collect and use evidence. The panelists also discussed what needs to happen next to further strengthen the use of evidence in the federal government. 

The roundtable featured remarks from Erika Rissi, Nick Hart, Melinda Buntin, Rachel Snyderman, and Paul Decker. 

Rissi is the chief evaluation officer of the National Science Foundation, where she is also the Head of its Evaluation and Assessment Capability Section.

Hart is the president and CEO of the Data Foundation. 

Buntin is a professor at the Johns Hopkins Bloomberg School of Public Health and the Carey Business School. 

Snyderman is the managing director of the Bipartisan Policy Center’s Economic Policy Program. 

Decker is the president and chief executive officer of Mathematica. 

You can also read a short blog by Decker summarizing key themes from the roundtable here: https://mathematica.org/blogs/advancing-and-enhancing-evidence-based-policymaking

Listen to a past episode of On the Evidence featuring a discussion about the five-year anniversary of the Evidence Act with experts from government, think tanks, research organizations, and academia: https://mathematica.org/blogs/building-on-the-evidence-act-to-increase-the-positive-impact-of-federal-policymaking

Listen to a past episode of On the Evidence featuring Robert Shea, an expert on performance improvement in government who served on the bipartisan U.S. Commission on Evidence-Based Policymaking, a group whose recommendations informed the Evidence Act: https://mathematica.org/blogs/robert-shea-on-the-federal-governments-progress-in-using-evidence-to-improve-programs-and-policies

135 | It’s the Evidence, Stupid: GAO’s Report on Evidence-Based Policymaking and What Comes Next

On the latest episode of Mathematica’s On the Evidence podcast, M-KOPA Chief Executive Officer Jesse Moore talks about increasing access to financial and digital services for unbanked workers in Africa by leasing out smartphones. He and Paul Decker, the president and chief executive officer of Mathematica, discuss the needs of everyday earners in Africa who work regular but informal jobs that make it difficult to build credit and open a bank account. They also discuss the role of data in informing an organization’s business strategy as well as assessing its social impact on customers. 

Find a blog summarizing highlights from the conversation, plus the full transcript here: mathematica.org/blogs/how-a-fintech-company-helps-workers-in-africa-access-credit-and-savings-with-smartphones

136 | How a Fintech Company Helps Workers in Africa Access Credit and Savings with Smartphones

On the latest episode of Mathematica’s On the Evidence podcast, Dr. Agnes Kalibata reflects on her career as a scientist and as the former president of the Alliance for a Green Revolution in Africa (AGRA), an African-led organization that seeks to create an environment where Africa can sustainably feed itself. It does so by focusing on scaling agricultural innovations that help smallholder farmers achieve increased incomes, better livelihoods, and improved food security.

Kalibata’s 10-year term as AGRA president ended earlier this year. She spoke with Mathematica President and Chief Executive Officer Paul Decker last December. Mathematica supports AGRA’s implementation of its 2023–2027 monitoring, evaluation, and learning strategy. On the episode, Kalibata and Decker discuss locally led development and the role of data in helping to understand how a program, such as AGRA’s Seed Systems, can be more effective.

Find a full transcript of the conversation here: https://mathematica.org/blogs/increasing-the-resilience-of-african-smallholder-farmers

Learn more about Mathematica’s work supporting AGRA as AGRA implements its 2023–2027 monitoring, evaluation, and learning strategy: https://mathematica.org/news/measuring-agras-impact-transforming-agricultural-systems-and-improving-climate-resilience

137 | Increasing the Resilience of African Smallholder Farmers

A podcast hosted by Mathematica’s J.B. Wogan that examines what we know about today’s most urgent challenges and how we can make progress in addressing them. Reimagining the way the world gathers and uses data, Mathematica uncovers the evidence that offers our partners the confidence and clarity they need to find out what can be done, how to make it happen, and where to go next.

100 | Robert Shea on the Federal Government’s Progress in Using Evidence

On the Evidence

Related tracks