Technology and policy for changing the world
By: Scott Murray
The 2024 MIT Policy Hackathon empowered participants with the data and methods to collaboratively tackle challenges in healthcare, climate resilience, and more.
Pulse oximeters are devices that provide critical data for medical decision-making. Though recent studies have surfaced a key limitation — they provide less accurate readings for patients with darker skin — official metrics for evaluating their performance haven’t fully kept pace with these new findings.
This is precisely the kind of problem the MIT Policy Hackathon was designed to tackle.
IDSS students from the Social and Engineering Systems (SES) PhD program and the Master’s-level Technology and Policy Program (TPP) organize the annual MIT Policy Hackathon, a weekend-long event that challenges teams to develop and propose data-informed policy solutions to real challenges sponsored by organizations from different sectors.
“The MIT Policy Hackathon convenes participants from across disciplines and sectors and challenges them to address solutions to pressing societal challenges through data-driven solutions, sociotechnical perspectives, and innovative policy strategies,” said TPP Director Christine Ortiz, the MIT Morris Cohen Professor of Materials Science and Engineering. “The hackathon employs immersive research, development, and collaboration, where teams work with stakeholders from local organizations and governments, to build relationships, advance solutions, and inform real-world policymaking.”
Health equity
Pulse oximeters are easy to use devices which use light to measure the saturation of oxygen in someone’s blood. Though their limitation may seem like merely a technical issue, policy has a crucial role to play in how the devices are used, with downstream consequences that can lead to disproportionate harm to racial and ethnic minorities that already suffer from disparities in health outcomes.
“The reliability of pulse oximetry readings is both an engineering and an equity issue,” said Marie-Laure Charpignon, an IDSS SES PhD alum whose doctoral research informed the challenge. Charpignon provided data for Hackathon participants from her collaborative research with Leo Celi (MIT HST) and Ian Wong (Duke), and served alongside Wong as a judge of the Health Equity challenge policy proposals. “Overall inaccuracy may seem low, but inaccurate readings in certain patient subgroups have downstream effects on treatment decisions. Falsely high readings can mislead oxygen therapy interventions to improve blood oxygenation, ultimately resulting in higher in-hospital mortality rates among Black and Hispanic patients.”
“There is more that we can be doing to better evaluate pulse oximeter effectiveness before market release during the regulatory approval process, and to facilitate clinical practice with easy-to-use methods that reduce unnecessary harm,” adds Charpignon.
Winning team OxEquity approached this challenge by analyzing patient data, including comparing oxygen measurements based on blood tests against pulse oximeter readings, to identify rates of misclassification within different demographic subgroups. The team brought together three graduate students from the Harvard T. H. Chan School of Public Health (Caleb J. Kumar, Muhammad Jawad Noon, and Xiao Wei Yeap), as well as Jane E. Valentine, a biomedical engineer at Johns Hopkins, and Yajie Xu, a doctoral student in economics at the University of South Florida.
“The Accuracy Root Mean Square (ARMS) metric currently used by the FDA to evaluate pulse oximeter performance focuses on general performance by providing an overall measure of average error,” reads the team’s policy brief. “In the regulatory context, pre-market studies submitted to the FDA often lack specific demographic representation because current guidelines do not mandate it. We identify rates of misclassification within each subgroup … and count the problematic misalignment proportion (PMP), as a fraction of overall cases.” Using the PMP metric, the team was able to draw attention onto the patient subgroups most affected by misleading pulse oximetry readings.
The team’s recommendations include adopting a weighted ARMS metric that accounts for the very differences in current device performance they identified, updating existing evaluation methods by quantifying device performance at the demographic subgroup-level, and requiring device manufacturers to to include minimum numbers of participants from high-risk subgroups in their pre-market studies.
For event judges, one key feature of a winning plan is viability. “OxEquity’s winning proposal demonstrated thoughtful data analysis and prioritized equity, while considering clinician and hospital burden,” said Charpignon. “They also developed a plan that is accessible and actionable, requiring less political capital to implement in the clinic.”
As the team brief concludes: “By building upon existing processes and integrating these changes incrementally, the FDA can make a substantial impact on health outcomes for underserved populations without imposing undue burdens on manufacturers or healthcare providers.”
Climate resilience
Another 2024 Hackathon challenge focused on local climate change risk, with a particular emphasis on impacts to some of Massachusetts most vulnerable communities. Sponsored by the Commonwealth of Massachusetts Executive Office of Energy and Environmental Affairs (EEA), in partnership with the Office of Climate Science (OCS) and the Department of Conservation and Recreation (DCR), the challenge was titled ‘Integrating Climate Resilience into Massachusetts Infrastructure and Land Use: A Water-Smart Approach to Equitable Growth.’
“Massachusetts faces significant climate risks, including increased flooding, droughts, and water infrastructure challenges,” reads the challenge brief prepared by the sponsors. “These risks threaten critical housing and water infrastructure … essential for public safety, economic stability, and the well-being of communities across the state. Vulnerable communities, particularly those facing housing and water affordability and access issues, are often the most impacted.”
The challenge for teams was to propose policy solutions that not only support water-smart growth in MA that is resilient to climate risks, but also to ensure equitable outcomes, particularly for low-income and underserved communities. To help develop these proposals, teams had access to multiple data sets across 4 categories:
- Climate, including projected temperature and precipitation statistics, floodplain maps, and environmental risk assessments
- Equity, including housing demographic data and information on groups identified as ‘environmental justice populations’
- Water Resource Availability, such as watershed capacity and protected drinking water locations
- Water Infrastructure, including spatial data on sewers and storm drains
“By addressing this challenge, Massachusetts has the opportunity to set a new standard for how state governments and local communities can collaborate to create equitable water-smart growth strategies that protect all residents in the face of climate change,” say the challenge sponsors.
The winning team, dubbed ‘HydroHomies,’ was composed of biomedical informatics researcher Bingrui Li, science policy researcher Kush Patel, and two quantitative traders: Andrew Bogdan and Tejas Saboo, who is now a biomedical cancer researcher. Their policy proposal focused on leveraging the success of the Local Initiative Program (LIP), a state program that provides technical assistance to incentivize affordable housing development, to promote broader water-smart development practices.
First, the team suggested using the available data to identify development zones that are “proactive” areas conducive to water-sustainable development, as well as “reactive” areas prone to the adverse impacts of climate change. Second, they recommend including a water readiness assessment as a component in awarding grants to affordable housing developers, and detailed a methodology using water availability and climate predictions. This approach makes the connection between water-smart growth and equity more explicit to ensure that all communities are prepared for the future.
The team provided additional suggestions for improvement of their proposal, such as the inclusion of additional data on water use, population, and even pollution. They are engaging with the Massachusetts Office of Energy and Environmental Affairs to improve the policy memo, and have been invited to present the proposal to the Massachusetts Water Resource Commission in 2025.
“Our team enjoyed the hackathon,” said Saboo. “The experience has actually inspired me to apply to graduate programs that integrate quantitative analysis, social science, and policy design to continue similar work.”
Engaging in the science and technology-policy lifecycle
The other challenges at the 2025 Policy Hackathon explored issues in housing and criminal justice.
The housing challenge was sponsored by Assembly OSM, a modular building company aiming to introduce new automated construction technology in the US real estate market. Teams used data on a real 15-story, mixed use residential building currently being developed in East Harlem, New York, as well as supply chain and open source data, to identify obstacles to innovation in the current housing policy landscape, and propose changes to help scale next-generation housing construction technology.
“We are in a really interesting moment for our industry,” said Assembly CEO Andrew Staniforth. “We have a massive housing crisis, a ticking clock to address climate change, and shifts in the labor market. This creates amazing tailwinds for new methods and it just takes a few players to completely change the industry.”
The criminal justice challenge, sponsored by Redo.io, with support from the LexLab UC Law SF 2024 Justice Tech Accelerator, asked participants to use California state prison incarceration data to develop new methods for identifying individuals harshly sentenced for low-level offenses. The winning team proposed an online tool to aid district attorneys and the public, combined with minimal mandates and a self-sustaining incentive system to hold district attorneys accountable.
“Each year, the challenges explored at the Policy Hackathon showcase real issues in a number of engineering and science domains related to policy,” said Ortiz. “And each year, the sponsors of those challenges are provided with fresh and dynamic ideas that can inform their approaches to those challenges.”
For event organizers like SES student Eric Liu, one of the Policy Hackathon’s goals — and one of its strengths — is engaging participants in the science and technology-policy lifecycle. “You have to be able to brainstorm without getting too caught up in judgment, but produce a plan that is viable. It can be messy and difficult to push through barriers. I think the experience helps show how all this is a part of the process, and not a sign of failure.”
At the same time, organizers never lose sight of the need for equitable solutions, and see the Hackathon as a way to deepen MIT connections with marginalized groups that often bear the outsized impacts of societal dilemmas. These avenues have also served as sources of recruitment to IDSS academic programs and online courses.
“It’s hard to quantify,” says Liu, “but the hope is to break down walls and help people feel that they belong here. They have the skills the world needs, and we want to give them the tools to change it.”
