
26 Nov CNEP Delivers Annual Evaluation Report and Highlights Challenges in Regulatory Impact Reports (RIR) Coverage
- The Annual Evaluation Report, presented today by CNEP, marks the first edition of an annual report aimed at systematizing, reviewing, and analyzing the performance of Chile’s public policy evaluation system, highlighting key advances and challenges in the field.
- This year, in addition to including reports on program and investment evaluations (2023) provided by DIPRES (Budget Directorate of the Ministry of Finance) and the Social Evaluation Subsecretariat (SES) of the Ministry of Social Development and Family (MDSF), CNEP’s analysis encompasses other areas and lines of evaluation present in Chile. These include Regulatory Impact Reports (RIR), which accompany all Executive-initiated bills and supreme decrees with regulatory impact.
- During 2023, out of 62 bills introduced via presidential message (subject to RIR), only 9.7% (6 out of 62) had a completed report, and 1.6% (1 out of 62) had one in progress.
- Additionally, 9 finalized RIRs during 2023 accompanied decrees. Comparing the total number of finalized RIRs with the total number of decrees submitted (145), coverage of this type of regulation reached 10.3% during the year.
- This year’s edition includes a chapter on the country’s Territorial Planning, addressing urgent areas and challenges.
- Even though the National Territorial Development Council (formerly the National Urban Development Council) in 2018 proposed concrete steps toward an integrated system, Chile still lacks such a framework. Additionally, challenges in preparing Territorial Planning Reports (TPRs) are identified, including the absence of long-term objectives and vision.
- Following heavy rains in 2023, several buildings in Viña del Mar suffered damage (e.g., a broken water collector), and parts of the area were declared uninhabitable. Why were so many buildings allowed constructionin a dune zone, exposing people to significant socio-natural risks? The Municipal Works Directorate stated that these buildings were authorized under the municipality’s regulatory plan, in force since 2002, and approved without undergoing a strategic environmental assessment, a process now mandatory for the preparation of Territorial Planning Reports (TPRs).
Monday, November 25, 2024
Chile has established a robust framework for evaluating public programs through the Program Monitoring and Evaluation System (SMyEP). This system, administered jointly by the Budget Office (DIPRES) under the Management Control Division of the Ministry of Finance and the Social Evaluation Subsecretariat (SES) of the Ministry of Social Development and Family, provides a methodological foundation for assessing program effectiveness.
The SMyEP operates dynamically, selecting programs for evaluation based on their life cycle stages. These stages, commonly referred to as ex ante (design evaluation), ex dure (monitoring), and ex post (results evaluation), align with conceptual, strategic, and methodological criteria to ensure comprehensive assessments.
In 2023, ex ante evaluations focused on newly designed programs or those with significant modifications. Of these, 57% received favorable design ratings, while 43% were flagged for technical shortcomings. This phase is critical for identifying and addressing design inconsistencies or weaknesses before implementation. The most common issues were related to evaluating outcomes, addressing coverage gaps, and improving accessibility.
Ex dure monitoring, conducted annually, tracks the performance and implementation of public programs based on fiscal execution data from the previous year. SES monitors social programs, while DIPRES oversees non-social programs. This phase aims to provide timely, relevant information for budget formulation and is closely linked to both ex ante and ex post evaluations. In 2023, 700 programs were monitored, with associated expenditures totaling CLP $33 trillion, approximately 13% of Chile’s GDP and 40% of the 2024 fiscal budget.
The number of monitored programs has grown significantly since the mid-2010s, with averages rising from 518 annually between 2012 and 2015 to 674 between 2016 and 2022. Despite this progress, SES and DIPRES identified significant fragmentation in programs, particularly in economic growth and education. Recommendations included improving coordination, reviewing overlapping programs, and merging initiatives to increase effectiveness and reach.
Performance monitoring revealed that 73% of non-universal programs adequately targeted their intended populations, while 23% had deficiencies. Regarding prioritization, 59% met expectations, while 37% required improvements. In terms of efficacy, 79% of programs had well-defined indicators to measure their objectives, while 20% exhibited weaknesses. Administrative costs were appropriately estimated in 77% of programs, but 21% showed inaccuracies, and 2% lacked sufficient data.
In 2023, DIPRES completed 16 ex post evaluations, covering a range of sectoral and program-specific assessments. Programs evaluated under the Focalized Domain Evaluations (EFA) and Government Program Evaluations (EPG) showed varied results: five with poor performance, three with low performance, four with medium performance, and one with high performance.
A sectoral evaluation of the “Choose to Live Healthy” (EVS) program revealed significant gaps in addressing structural and individual factors related to non-communicable diseases. The program lacked coordination, monitoring, and long-term planning, limiting its effectiveness. DIPRES has established commitments with agencies to address these deficiencies. Of the 38 programs with commitments, five fully resolved their issues, 11 made progress but have pending commitments, 18 partially addressed concerns, and four failed to comply.
Public Investment Evaluations
Public investments—such as roads, bridges, hospitals, schools, transportation systems, social housing, urban improvements, and healthcare services—are assessed and monitored by the National Investment System (SNI). This system operates under the joint administration of the Social Evaluation Subsecretariat (SES), part of the Ministry of Social Development and Family (MDSF), through its Division of Social Investment Evaluation (DESI), and the Budget Office (DIPRES), which is responsible for the formulation and execution of investment project budgets.
Established in 1960, the SNI is the country’s oldest evaluation framework. It is built on a structured set of norms, procedures, and criteria designed to regulate and coordinate the complete investment cycle—from project identification and formulation to evaluation, prioritization, and resource allocation. Its primary goal is to optimize the use of public funds, ensuring that projects are socially profitable and aligned with national development objectives.
A robust methodology underpins the SNI, defining parameters for ex-ante evaluations to ensure that all initiatives undergo rigorous financial viability and social profitability assessments before securing funding.
Key milestones in the investment evaluation process include ex-ante evaluation, budget formulation and execution, and ex-post evaluation.
In 2023, 6,563 investment initiatives were submitted for ex-ante evaluation under the SNI, representing a combined investment of USD 41.6 billion. Of these, 67% received a satisfactory recommendation, 9% faced technical objections, and 23% were flagged for insufficient information.
Municipalities accounted for the highest amount of technical objections (58% of initiatives), compared to ministries, which recorded a lower rate of 33%.
The ex-post evaluation within the SNI analyzes the outcomes of completed projects, beginning with a strategic selection of initiatives. Depending on the timing of the evaluation, projects are categorized as short-, medium-, or long-term.
Short-term evaluations reveal that while 44% of completed projects are eligible, only 19% undergo evaluation. Comparing execution outcomes with initial estimates uncovers significant issues in the formulation phase that were not flagged as risks during evaluation, leading to notable cost overruns and schedule deviations.
In 2021, cost inaccuracies reached 56% for ministry projects and 49% for municipal projects. Compared to 2020, these figures increased for ministries and decreased for municipalities.
In terms of project timelines, deviations reached 80% for ministries and 76% for municipalities. Although these indices have remained relatively stable since 2016, the significant gap between actual outcomes and initial projections remains a pressing concern.
The authorities supplement the publication of evaluation data with a summary report highlighting results. However, the feedback loop between these findings and the ex-ante evaluation phase remains unclear, raising questions about how identified risks are addressed in future project formulations.
In 2023, medium-term ex-post evaluations focused on the Regional Health Networks of Antofagasta and Magallanes, assessing the capacity and performance of 11 existing projects. Preliminary findings identified healthcare gaps relative to estimated demand, including (i) significant isolation and distance from healthcare centers, (ii) shortages of medical equipment for preventive care, (iii) fragmentation across the integrated health network (primary, secondary, and tertiary levels), (iv) challenges in coordination and IT management, (v) lengthy project timelines, (vi) mismatches between demand estimates and infrastructure execution, and (vii) critical shortages of medical specialists.
To address these challenges, the evaluation recommended several measures, including implementing IT systems within hospital units, incorporating Family Health Centers (CESFAMs) into hospitals, improving demand estimation through epidemiological analyses, enhancing the capacity of low-complexity centers, and increasing infrastructure investments, particularly in hospital units.
Significant reforms to SNI norms, procedures, and methodologies were also implemented in 2023, aimed at strengthening the system and enhancing public investment evaluations in Chile.
The CNEP emphasizes the need to revisit inclusion criteria for public investment projects, advance the application of proportionality principles in evaluations, review project selection criteria for ex post evaluations, and bolster the system’s feedback mechanisms to ensure continuous improvement and effectiveness.
Regulatory Impact Assessments (IIRs)
Chile’s regulatory framework, while lacking a consolidated evaluation system comparable to those used for public programs and investments, has seen the emergence of various independent tools in recent years that influence the evaluation of public policies. These include Regulatory Impact Reports (IIRs) and ex post evaluations conducted by the Department of Law Evaluation within the Chamber of Deputies.
Introduced in 2019, Regulatory Impact Reports (IIRs) are managed by the Ministry of Economy, Development, and Tourism (MINECON) and the Ministry General Secretariat of the Presidency (SEGPRES). These reports are mandatory for all legislative bills initiated by the Executive branch, as well as Supreme Decrees issued either directly by the President of the Republic or by a minister acting “on behalf of the President.”
IIRs operate under the principle of proportionality, meaning the scope and depth of analysis correspond to the estimated impact of the regulatory change. Depending on the regulatory context, IIRs can include up to three components: Preliminary Evaluation (EP), Standard Regulatory Impact Report (IIR-E), and High-Impact Regulatory Impact Report (IIR-AI).
While the IIRs included in this inaugural edition of the CNEP Annual Evaluation Report were prepared in 2023, many pertain to legislative bills (PdLs) introduced in prior years. Of the 17 IIRs reviewed, seven were linked to PdLs submitted in 2023, of which six have been finalized and one remains in progress.
In 2023, 62 legislative bills were submitted via presidential messages, all of which required IIRs. However, only 9.7% (6 out of 62) had completed reports, with 1.6% (1 out of 62) still in development.
For decrees, 9 IIRs were finalized in 2023, including seven preliminary evaluations and two standard reports. When comparing the total finalized IIRs with the 145 decrees issued during the year, the coverage reached only 10.3%.
The CNEP emphasizes that increased dissemination of these reports and their findings would not only enhance transparency in regulatory processes but also strengthen Chile’s evaluation culture. Greater visibility and understanding of the results could promote stronger accountability and evidence-based decision-making, ensuring that regulatory policies are both effective and impactful.
National Evaluation and Productivity Commission (CNEP)
This annual report also includes topics related to the evaluations conducted by the CNEP. While the commission is a key player in Chile’s evaluation system, the reports it produces do not fall neatly into one of the previously mentioned systems. Depending on the mandate it receives, the CNEP’s evaluations may focus on programs, investments, and/or regulations.
In 2023, the CNEP completed the study titled “Analysis of Priority Sectoral Permits for Investment”, which constitutes a regulatory evaluation focused on improving the efficiency of these regulations.
The study, covering the period between 2018 and 2022, identified 439 administrative procedures affecting the investment process, prioritizing those that could act as barriers. It identified 71 entities responsible for issuing 309 permits required to transform a project into an investment, of which 63 (the focus of the study) were classified as critical, as they have the potential to halt an investment.
The analysis revealed significant issues in Chile’s current permit system, including:
- Inefficiency: Excessively long response times from the state.
- Instability: Authorizations granted for investments may later be revoked by another government agency.
- Unpredictability: Approval requirements for permits vary across institutions and over time, without clear criteria.
The key finding was that the permits required for investment projects in Chile form a heterogeneous system, whose behavior is closely linked to the level of complexity of the permits.
Among the proposed recommendations are:
- Establishing an enabling governance model to increase transparency and define the scope of permits.
- Implementing cost transfer mechanisms to users.
- Increasing digitalization to streamline processes, including those managed by the Council of National Monuments.
- Simplifying and expediting permit issuance procedures.
It is worth noting that one of the measures currently under parliamentary discussion is a comprehensive reform of the permit system, incorporating the insights and recommendations provided by the CNEP’s analysis.
Territorial Planning Challenges
Interest in the impact of Territorial Planning Instruments (IPTs) in urban areas or regions with human settlements has been steadily increasing in Chile. A recent case illustrating the consequences of these policies is the so-called “sinkhole case,” which occurred in the winter of 2023 in the Viña del Mar commune. Several buildings sustained damage due to the collapse of a stormwater collector, exacerbated by heavy rains.
The failure of the collector posed an imminent danger to nearby residents, prompting municipal authorities to declare certain buildings uninhabitable and proceed with evacuations. This incident raised a critical question: why was the construction of so many buildings permitted in a dune area, exposing people to significant socio-natural risks? The Municipal Works Directorate revealed that these constructions were authorized under the commune’s regulatory plan, in effect since 2002, which had been approved without undergoing a strategic environmental assessment—a process now mandatory for IPTs.
In Chile, IPTs are primarily regulatory documents that establish binding rules to organize and regulate land use, especially in cities or inhabited areas. While their development is strictly governed by current legislation, there is near-universal consensus that both their creation and modification are arduous and lengthy processes. Recently disclosed data from the Ministry of Housing and Urban Development (MINVU) indicate that it takes an average of six to ten years to develop a Communal Regulatory Plan (PRC).
While Communal and Sectional Regulatory Plans are created by municipalities, intercommunal plans fall under the responsibility of the MINVU’s Regional Secretary and are implemented by regional governments. Although legislation mandates that every commune or intercommunal area must have its own IPT, many territories either lack a valid plan or have outdated ones. Recent findings show that only 54 (13% of all communal IPTs nationwide) have been updated in the past decade, corresponding to just 21% of Chilean communes. Currently, 148 modifications to communal IPTs are in progress, along with the development of 42 new instruments.
Chile Lacks an Integrated Planning System
The CNEP’s analysis highlights that IPTs in Chile are overly regulatory in nature, lacking clear objectives and long-term vision, which complicates their evaluation. This issue stems from the absence of an integrated planning system. In 2018, the National Territorial Development Council (formerly the National Urban Development Council) proposed a concrete framework for advancing toward such a system.
An integrated planning system would facilitate the evaluation of IPTs by providing clearer parameters for assessing regulations. Drawing from international experiences, the CNEP noted that Colombia, for example, has instruments that address not only local territorial planning but also broader objectives, such as long-term community goals, rural development, environmental protection, disaster risk prevention, housing, and transportation.
To make PRCs robust instruments, the CNEP identifies two possible approaches: creating integrated plans that encompass development, urban planning, and management elements, or maintaining these as separate instruments but ensuring they are interconnected.
Other shortcomings include a lack of coordination between different planning levels, unclear standards that complicate evaluations, and high complexity and costs associated with developing new regulatory plans. Additionally, the lack of standardization in publishing IPT documents hinders their understanding and evaluation, resulting in limited accessibility and inconsistent content.
Recommendations for Improvement
To address these and other challenges, the CNEP has proposed a series of recommendations. While some require structural changes to urban planning legislation, others could be implemented without altering the current legal framework.
Among those not requiring legislative changes, the CNEP suggests establishing incentives and reporting mechanisms to ensure IPTs are developed and updated in closer coordination with other territorial planning instruments. The report also recommends setting measurable targets for regional and local governments during IPT development processes, administratively mandating monitoring and reporting duties, and fostering partnerships with academic institutions to produce rigorous, independent, and objective studies on the impact of these instruments.
For proposals requiring structural legal modifications, the recommendations include advancing toward an integrated planning system in line with the National Territorial Development Council’s proposals, defining clearer hierarchies and coordination between IPTs and other planning mechanisms, and legislating specific evaluation and reporting requirements for IPTs.
In this first edition of its Annual Evaluation Report, the CNEP concludes that further progress in systematizing information across all addressed areas is essential to optimize feedback processes.