Innovations for environmental compliance: emerging evidence and opportunities
Key Takeaways
- Environmental practices have improved significantly over the last half century, but noncompliance with environmental law remains stubborn.
- New data sources and machine learning are powerful tools to assess risks, enhance enforcement and improve compliance.
- Learning how to best combine machine learning with compliance interventions will require collaborative partnerships to rigorously pilot and evaluate impacts.
Environmental law aims to protect the air, water, and land that our communities are built around. Despite significant improvements in environmental practices over the last half century, noncompliance with U.S. environmental regulations remains pervasive. Tens of millions of Americans remain exposed to unsafe drinking water and pollution hotspots where exposure to toxic emissions increase health risks (Allaire et al. 2018; Gallay 2019).
Some of the best evidence on the extent of noncompliance comes from a series of large-scale, randomly-sampled evaluations conducted by the U.S. Environmental Protection Agency in the early 2000s. The results showed that, for example, 61 percent of municipalities with combined storm and wastewater systems failed to maintain adequate minimum controls for managing overflows. Overflows in these systems can include releases of untreated stormwater, human and industrial wastes, toxic materials, and debris. Separately, over the last eight years, 60 to 75 percent of the nearly 7,000 major facilities with Clean Water Act permits — including the largest and highest impact power plants and wastewater treatment facilities — have reported they are not in compliance with federal law.
The standard economic explanation of compliance hinges on deterrence: Facilities are deterred from violating environmental regulations if compliance is cheaper than the expected sanction. This deterrence model emphasizes two policy levers: the frequency of inspections to enhance the probability of detection and the magnitude of sanctions.
While inspections and sanctions are critical to improve compliance, resources for enforcement have diminished over the past decade. For instance, federal inspections fell from nearly 22,000 in 2007 to 12,000 in 2017 — a decline of 45 percent. Budgetary and political constraints suggest inspections and enforcement actions will not increase substantially anytime soon. Although inspections are constrained, technological advances and behavioral insights can help transform what some describe as a game of whack-a-mole into a forward-looking, tailored system to identify and target critical risks.
This policy brief highlights emerging research on interventions to improve environmental compliance. We show evidence from several fields that suggests how regulators can leverage powerful new sources of information, data science, and behavioral insights to promote environmental compliance.
These tools include revamping detection, lowering compliance costs, and increasing public disclosure. All of these tools fall in a class of “information-based” alternative enforcement techniques that emerging research suggests as promising, but will require adaptation, piloting, and testing to deploy effectively.
Better detection through machine learning
Rapid advances in machine learning enable regulators to dramatically lower the costs of monitoring and detection by extracting insights from rich new sources of data. Similar to how search engines can predict the most relevant advertisements, machine learning can predict which facilities pose the highest risk of failing an inspection and pinpoint the areas of greatest environmental risk.
A common approach to machine learning involves training a model on large numbers of observations to “learn” features or patterns associated with a given outcome. When these data-driven models are combined with local information on which types of violations matter most, these approaches can help inspectors learn from experiences beyond their own jurisdiction and focus resources on the facilities with the highest impact.
A report for the Administrative Conference of the United States found that enforcement is the typical use case for machine learning in federal government (Engstrom et al. 2020), but environmental regulators have been slow to adopt these promising techniques. In recent work, Hino, Benami, & Brooks (2018) demonstrate how using machine learning techniques on over 300,000 facilities regulated by the Clean Water Act could double the number of violations detected without increasing inspections. Their approach uses data on violation histories and facility characteristics to predict risk and then direct inspections towards the riskiest facilities. Other agencies have similarly used machine learning to prioritize the processing of complaints and investigative resources.
A particularly active area of machine learning is in “deep learning,” which uses big data and complex models to detect potentially subtle patterns. Since 2011, deep learning has revolutionized areas of image and speech processing. In the context of environmental enforcement, such advances can enable faster, cheaper, and continuous evaluation of high-resolution satellite imagery to detect noncompliance problems.
For example, although ambient water quality monitors show agricultural waste contributes to significant water impairment, many agricultural operations have remained both unpermitted and unmonitored. Handan-Nader & Ho (2019) demonstrated that deep learning techniques can identify 95 percent of sources of pollution at 10 percent of the resources required for a comparable manual approach.
Handan-Nader, Ho, & Liu (2020) further describe a number of other currently-underutilized ways that satellite imagery analysis can support environmental monitoring, such as wetland conversion, land use violations, and habitat modification.
Risk notification
These advances in detection can further promote compliance when communicated to facilities. Alerting facilities that they are at “high risk” of a violation can signal that those facilities are being observed and can thereby increase the perceived likelihood of detection.
Field experiments demonstrate that regulated entities respond to messages conveying an increased probability of detection. For example, Danish taxpayers randomly selected to receive a letter indicating a 50 or 100 percent audit probability increased reported income by 1.1 and 2 percentage points, respectively (Kleven et al. 2011). And those individual results may generalize to organizations. Randomly-selected small and medium-sized firms in Uruguay increased tax payments by 6.4 percent after receiving letters about audit frequency and penalty rates (Bergolo et al. 2019).
That said, risk notifications may be ineffective if parties do not change their beliefs about detection and penalties. Norwegian manufacturers reduced hazardous waste violations 15 to 23 percentage points following an audit by the Norwegian EPA but did not reduce violations (relative to a control group) when notified of increased audit frequency (Telle 2013). Due to extensive use of warnings before the trial, firms may simply not have believed or paid attention to the trial’s message.
Improved detection methods enable regulators to enhance the salience and credibility of messages indicating that the chances of detecting a violation have increased. Risk notifications that include information about enforcement activities at peer facilities may also be particularly effective (Shimshack 2014). Surveys of compliance managers and studies on the impacts of enforcement actions find that facilities often change environmental practices in response to inspections and penalties at peer facilities.
Prompting better compliance
In addition to changing the calculus of detection, data science can empower regulators to lower the cost of compliance.
Basic deterrence theory assumes that (1) all expected benefits and costs of taking an action by the regulated entity are known and (2) collecting such information is costless. Such assumptions are likely violated in the everyday context of operating complex facilities such as wastewater treatment and industrial manufacturing plants. Regulated entities may be uncertain about legal requirements, precluding cost-benefit analyses of possible compliance activities, or might lack the internal management capabilities necessary to undertake such evaluations.
Such challenges are especially prevalent among small facilities with few dedicated compliance staff as well as facilities facing complex or frequently-changing regulations.
Although regulators may be unable to address severe capacity or financial constraints, regulators can translate data-driven insights into actionable information about how to comply that lowers facilities’ learning costs:
- Self-inspection assessments. Distilling complex regulatory obligations into a series of clear key compliance indicators is one way to help reduce knowledge gaps and clarify critical actions. For example, starting in 2007, Colorado required that small hazardous waste generators complete and return a compliance self-certification. Over three years, compliance rates increased from 32 to 84 percent. Such efforts appear particularly appropriate for a sector where on-site inspections could at best reach only 12 percent of facilities per year and where the costs of compliance actions were relatively low.
While mandatory self-assessments appear promising, it remains unclear if the cause for their success is in the checklist itself or the associated training and organizational or cultural change (Martinez et al. 2015; Gibbons & Kaplan 2015; Ho et al. 2018). Designing interventions to isolate the cause for these results is important for informing how to focus future compliance strategies.
- Tailored technical assistance. While evidence on assistance interventions is limited, some evidence suggests the EPA’s regional technical assistance activities, such as on-site visits, workshops, web-based assistance training programs, and guidance documents have helped decrease noncompliance among small- and medium-sized hazardous waste-generating facilities (Stafford 2012). Such technical assistance can prove especially fruitful for regulators overseeing numerous resource-limited facilities facing similar classes of challenges.
While the evidence base is weaker for technical assistance, data-driven techniques could help make such assistance more relevant. Models can forecast specific risk factors, enabling technical assistance to be offered where needed most. Data-driven techniques could also identify peer facilities that have overcome similar compliance challenges to speed up learning. One regional EPA office has begun to pair similar compliant and noncompliant wastewater treatment plants to jumpstart this type of knowledge exchange. Such interventions may signal the future of data-powered technical assistance.
Amplifying sanctions through disclosure
While traditional enforcement strategies focus on legal sanctions, public disclosure can amplify sanctions for noncompliance. And advances in data science and behavioral insights can help enhance public disclosure by tailoring and placing the information where most relevant.
“Naming and shaming” can trigger public and political scrutiny, damage reputations and lead to declines in financial market standing, especially for consumer-facing organizations. After Massachusetts mandated that drinking water suppliers notify customers about contaminant levels and violations via direct mail, violations dropped by a third to half.
After facility listings under the EPA’s Clean Air Act “Watch List” were publicized, the probability of a violation fell 10-23 percent. Separately, after the EPA announced stricter enforcement against electric power plants under the Clean Air Act, facilities at-risk of enforcement lawsuits reduced emissions significantly.
At the same time, the evidence base for disclosure remains mixed. Disclosure can induce gaming by firms and divert scarce regulatory resources (Ho 2012; Slemrod 2019). Many disclosures are simply ignored. To make them more effective, some research suggests that when states publicly released condensed summary information, rather than complex raw data, environmental performance at regulated facilities improved (Bae et al. 2010).
A key challenge for regulators lies in making these disclosures accurate, effective, and meaningful.
Where to from here?
Rapid advances in machine learning have begun to transform nearly all sectors across the economy.
While the approaches we highlight here hold tremendous promise, the most effective interventions demand adaptation to the environmental and regulatory domain. Developing the most effective compliance strategies with these approaches will require rigorous design, piloting, and evaluation in partnership among regulatory agencies, data scientists, behavioral scientists, and stakeholders. Regulatory agencies sometimes incorrectly assume that rigorous evaluation will be very costly. In contrast, we can phase new approaches to learn what works in ways that extend the reach of existing resources.
Much has been learned, but research-practice partnerships have the potential to create scalable and cost-effective methods that enhance existing efforts to address the stubborn problem of environmental noncompliance.
References
Allaire, Maura, Haowei Wu, & Upmanu Lall (2018). “National Trends in Drinking Water Quality Violations”. Proceedings of the National Academy of Sciences 115.9, pp. 2078–2083.
Bae, Hyunhoe, Peter Wilcoen, & David Popp (2010). “Information Disclosure Policy: Do State Data Processing Efforts Help More Than the Information Disclosure Itself?” Journal of Policy Analysis and Management 29.1, pp. 163–182.
Bergolo, Marcelo, Rodrigo Ceni, Guillermo Cruces, Matias Giaccobasso, & Ricardo Perez-Truglia (2019). “Tax Audits as Scarecrows: Evidence from a Large-Scale Field Experiment”. IZA Discussion Papers, No. 12335, Institute of Labor Economics (IZA).
Engstrom, David Freeman, Daniel E. Ho, Catherine M. Sharkey, & Mariano Florentino-Cuéllar (2020). Government by Algorithm: Artificial Intelligence in Federal Administrative Agencies. Administrative Conference of the United States.
Gallay, Paul (2019). “The Right to Know and the Responsibility to Act: Ensuring Environmental Compliance through Inspection, Enforcement and Citizen Science”. ABA Section on Environment, Energy, and Resources.
Gibbons, Robert & Robert S. Kaplan (2015). “Formal Measures in Informal Management: Can a Balanced Scorecard Change a Culture?” American Economic Review 105.5, pp. 447–451.
Handan-Nader, Cassandra & Daniel E. Ho (2019). “Deep Learning to Map Concentrated Animal Feeding Operations”. Nature Sustainability 2.4, pp. 298-306.
Handan-Nader, Cassandra, Daniel E. Ho, & Larry Y Liu (2020). “Deep Learning with Satellite Imagery to Enhance Environmental Enforcement”. Data-Driven Insights and Decisions: A Sustainability Perspective. Elsevier.
Hino, Miyuki, Elinor Benami, & Nina Brooks (2018). “Machine Learning for Environmental Monitoring”. Nature Sustainability 1.10, pp. 583-588.
Ho, Daniel E. (2012). “Fudging the Nudge: Information Disclosure and Restaurant Grading”. The Yale Law Journal. 122.3, pp. 522–851.
Ho, Daniel E., Sam Sherman, & Phil Wyman. (2018). “Do Checklists Make a Difference? A Natural Experiment from Food Safety Enforcement”. Journal of Empirical Legal Studies 15.2, pp. 242–277.
Kleven, Henrik Jacobsen, Martin B. Knudsen, Claus Thustrup Kreiner, Søren Pedersen, & Emmanuel Saez 2011). “Unwilling or Unable to Cheat? Evidence from a Tax Audit Experiment in Denmark”. Econometrica 79.3, pp. 651–692.
Martinez, Elizabeth A., Nancy Beaulieu, Robert Gibbons, Peter Pronovost, & Thomas Wang (2015). “Organizational Culture and Performance”. American Economic Review 105.5, pp. 331–335.
Shimshack, Jay P. (2014). “The Economics of Environmental Monitoring and Enforcement”. Annual Review of Resource Economics 6.1, pp. 339–360.
Slemrod, Joel (2019). “Tax Compliance and Enforcement”. Journal of Economic Literature 57.4, pp. 904–954.
Stafford, Sarah (2012). “Do Carrots Work? Examining the Effectiveness of EPA’s Compliance Assistance Program”. Journal of Policy Analysis and Management 31.3, pp. 534–555.
Telle, Kjetil (2013). “Monitoring and Enforcement of Environmental Regulations: Lessons from a Natural Field Experiment in Norway”. Journal of Public Economics 99.2, pp. 262–288.