Will Shifting English Learning Accountability to Schools Work?
Now that all fifty states and the District of Columbia have received formal federal approval of their Every Student Succeeds Act (ESSA) plans, the work of implementing the 2015 law—the United States’ primary K–12 education investment—is fully underway. As that work continues, however, advocates for educational equity should keep close eye on ways that ESSA’s structure creates new, unforeseen challenges for historically underserved students. This is particularly true as far as English learners (ELs) are concerned, given that ESSA made significant changes to how schools are held accountable for these students’ performance. Specifically, there is some evidence that ESSA’s shift of EL accountability from the district to the school level could make these students less of a priority for many schools.
Replacing No Child Left Behind
Like most of ESSA’s big changes, the law’s EL accountability reforms are a response to troubles with No Child Left Behind (NCLB), the 2001 law it replaced. Of course, these problems weren’t intentionally caused. No Child Left Behind was initially heralded as an advance for English learners in U.S. schools. Its Title III provisions provided new federal funding dedicated to serving ELs. The structure was intuitive: ELs had long been marginalized by—and in—U.S. schools. So Title III provided dedicated funding and focused accountability that would force districts to prioritize these children’s linguistic and academic development. Districts receiving Title III dollars were required to meet Annual Measurable Achievement Objectives (AMAOs): if a district’s ELs showed low academic and linguistic progress over a number of years, the district would face a range of state interventions designed to improve how it served those students.
This was a new policy experiment—ELs had never before had a stream of federal formula funding—and its limits soon became evident. While Title III was funded between $650 million and $750 million most years, it turned out that, when spread out across the country, this was hardly sufficient to focus local education leaders on ELs’ needs, particularly as the number of ELs in U.S. schools grew by approximately one million students. A national study of Title III implementation found some hesitation by state education officials when it came to imposing consequences on districts where ELs were chronically struggling. While Title III dollars and accountability put EL students on the map for schools and districts, many advocates felt that they were inadequate to making ELs’ development a top concern.
ESSA’s Solution…and a Possible New Problem
The Every Student Succeeds Act upends this status quo with an eye to addressing Title III’s shortcomings. It maintains Title III’s funding structure, but shifts the federal government’s primary EL accountability mechanisms into Title I, its primary K–12 funding stream. Title I funds support educational equity in schools serving large percentages of low-income students. Congress has funded it at over $15 billion annually in recent years. Moreover, Title I included NCLB’s signature provision: the goal that all students be proficient in reading and math by 2014, with consequences for schools and districts receiving Title I funds that fell short of annual progress toward that goal.
EL advocates argued that making ELs’ English acquisition a central part of the federal government’s largest stream of K–12 funding would lead more schools and educators to attend to ELs’ needs by including accountability checks for ELs’ progress in the government’s primary accountability system for monitoring the progress of all students. They convinced Congress: ESSA mandates that schools serving ELs will now be rated using a single system that takes into account traditional measures, like achievement in reading and math, alongside ELs’ annual progress learning English. The idea, as I noted at the time, was that ELs would get more attention if accountability for their performance was attached to that much larger funding stream. These funds—and the attached accountability systems—are too big for most schools to ignore.
That last bit is critical: ESSA’s Title I accountability requirements are applied solely at the school level, unlike the old Title III district-level accountability (importantly, NCLB’s Title I included both school- and district-level accountability requirements, but federal lawmakers removed the latter in ESSA). This poses a challenge for ESSA’s theory of action on EL accountability. While school-level accountability focuses pressure on the decision makers closest to these students, it could also—paradoxically—make ELs less of a priority for them.
This is because federal law requires states to set minimum student subgroup sizes for accountability calculations—the number of students enrolled in a school that is necessary in order to include a particular group in school ratings. These “n-sizes” are designed to support valid and reliable data collection. The smaller the size of a group of students, the more likely that their achievement numbers will fluctuate. N-size rules aim to stabilize data to be more consistent over time—especially when those data have accountability stakes attached.
N-sizes also protect students’ privacy. This should be intuitive: if there are just three or four ELs in a particular school, any sharing of that school’s EL achievement data in public accountability systems could run close to publicly revealing information about an individual student’s progress. So states also set a minimum n-size to protect students’ privacy—most are between ten and twenty-five students.
Here’s the key: If a school has fewer students in a particular subgroup than the state’s minimum n-size, that subgroup is left out of accountability calculations.
That’s why ESSA’s new approach to EL accountability could pose problems for EL advocates’ theory of action. Say, for instance, that a state sets its minimum n-size for subgroups at ten students. If an accountability system is calculated at the district level, that means that accountability provisions covering ELs’ academic and language acquisition progress only kick in when there are at least ten ELs enrolled in the district. If an accountability system is calculated only at the school level, however, that means that the accountability provisions only kick in when there are at least ten ELs enrolled in a particular school.
This isn’t a brand-new concern. Back in 2010, a study of national Title III implementation under NCLB suggested that n-size differences might have contributed to disparities in which districts were targeted for which kinds of improvement. Specifically, it noted that “districts with small populations may not have had enough ELs to qualify for a Title I accountability rating for the EL subgroup but did have enough ELs to qualify for a Title III accountability rating.” If n-sizes limited the inclusion of ELs at the district level under NCLB, it is almost guaranteed that n-sizes will limit the inclusion of ELs at the school level under ESSA—and compound the issue by drawing inequitable public attention to the problem of ELs getting inadequate educational opportunities.
Sign up for updates.
How Serious Is This?
So: in an effort to raise ELs’ profiles by moving federal EL accountability into Title I’s school-level accountability calculations, did advocates inadvertently reduce their prominence?
Let’s take a look, using a few state examples. Rough calculations using Louisiana’s publicly available district enrollment data from October 2013 suggest that over 99 percent of the state’s ELs were in districts with at least ten ELs. Given that Louisiana sets its minimum student subgroup size for accountability purposes (“n-size”) at ten students, those students would have been covered by No Child Left Behind’s Title III accountability provisions.
But if we apply the new ESSA accountability structure to this (admittedly old) dataset, things look different. According to the state’s school enrollment data from October 2013, around 85 percent of the state’s ELs attended schools with more than ten ELs. Under Louisiana’s ESSA plan, those schools would have their ELs’ progress towards English language proficiency counted in their overall accountability calculations.
In simpler terms: the new school-level EL accountability calculation would have removed over 2,000 ELs (15 percent of the state’s ELs) from Louisiana’s school accountability system that year.
Note, again, that this is an artificial scenario: in 2013, ESSA’s design and passage were still two years in the future (which is why I picked those data). To confirm the persistence of the problem, I reran the analysis with Louisiana’s 2017 numbers. That year, ESSA’s school-level EL accountability would have covered about 90 percent of the state’s ELs while NCLB’s old district-level accountability system would have covered over 99 percent. In 2017, at least 6,000 of Louisiana’s ELs were enrolled in schools that did not meet the state’s n-size—in other words, that served fewer than ten ELs (Note: there were many more ELs enrolled in Louisiana in 2017 than in 2013). Indeed, in 2017, two-thirds of Louisiana’s schools did not meet the n-size for EL enrollment.
This rudimentary analysis illustrates a real problem: moving EL accountability exclusively to the school level could make it more likely that ELs’ progress towards acquiring English language proficiency isn’t counted in schools’ accountability calculations.
Louisiana’s scenario is not unique. I ran the same analysis using Minnesota’s publicly available enrollment data (also from the 2013–14 school year). Minnesota set its minimum n-size for subgroups at twenty students; almost 99 percent of the state’s ELs were enrolled in districts with at least twenty ELs enrolled. By contrast, 95 percent of the state’s ELs are enrolled in schools with at least twenty ELs. In other words, the new approach under ESSA reduces the number of ELs included in school accountability ratings by over 3,000 children. In 2016–17, the proportions were almost identical.
What to Do?
None of this means that school-level EL accountability is fatally flawed. Indeed, advocates may be correct that those ELs counted under the ESSA’s approach to EL accountability will get significantly more attention than they would have under Title III’s district-level accountability system—and that benefit may be worth the cost of including fewer ELs in the accountability systems.
Fortunately, there are ways that states can maintain the focus of school-level EL accountability while navigating the challenge of overseeing schools with small numbers of ELs:
1. Calculate subgroup performance across multiple years.
The state of Washington’s ESSA plan illustrates one possible solution to the issue. The state set its ESSA n-size at twenty students. While developing its ESSA plan, it ran a series of analyses to see how different n-size policies would determine how many ELs would be included in the state’s accountability system. It published the results in the following table.
Comparison of Washington state’s proposed methodology to federal minimum guidelines. Source: Washington Superintendent of Public Instruction and U.S. Department of Education.
Look at the bottom row of the table. Just 63.5 percent of Washington’s ELs are enrolled in schools serving twenty or more ELs. By contrast, 82.8 percent of Washington’s ELs attend schools with ten or more ELs.
In response, Washington policymakers decided to set their n-size at twenty students, but allowed it to be calculated across three years. That is, if a school has 11 ELs one year, 14 ELs the next, and 10 ELs the third year, this would add up to a total of 35 data points, which would meet the state’s n-size (calculated across three years). The state found that this would include nearly 89 percent of the state’s ELs (see the cell in the bottom left corner of the table).
This may not reach the same degree of coverage as the state’s previous district-level EL accountability system, but it expands its range somewhat. Notably, a recent Alliance for Excellent Education analysis found that just nine states proposed to combine several years of data to ensure that more schools meet n-size rules for ESSA accountability.
2. Incorporate ELs’ progress towards learning English into other academic indicators.
In the ESSA plan it submitted to the U.S. Department of Education, Louisiana worried that its “minimum N size would cause the majority [of] schools to be excluded from EL progress monitoring.”
ESSA requires states to measure and rate schools on a range of metrics, not just on ELs’ progress acquiring English. Louisiana converts each of these elements—student growth and proficiency on academic assessments, high school graduation rates, and so forth—into points, so they can be combined into a single, overall accountability score for the school.
The state realized that this system gave it an option for addressing the n-size problem for school-level accountability. Despite setting its n-size at ten, Louisiana still found that over half of its schools wouldn’t meet that threshold and be covered by EL accountability. So it decided to convert ELs’ progress learning English into “points” that it could factor into their schools’ overall accountability scores. ELs attending schools with fewer than ten ELs wouldn’t have their English acquisition progress published, but it would still be counted towards the school’s overall accountability rating. In addition, ELs’ progress towards English proficiency counts more as the number of ELs in a school increases. That English acquisition metric’s share of a school’s score goes up as the school’s share of ELs increases.
In other words, every EL in Louisiana schools would be counted towards the school’s overall score. This sends exactly the signal that advocates hoped for: ELs’ English acquisition should be a priority for all schools.
3. Reinstate district-level accountability on top of school-level accountability.
It’s easy to think of new policies solely as alternatives to old ones. That is, if NCLB’s Title III EL accountability system focused on districts, and ESSA’s EL accountability provisions focus on schools, it can seem as though we have to choose. But there’s no substantive reason why U.S. public education can’t hold both schools and districts accountable for the same activity.
The advantages of a combinatory approach are obvious: a robust school-level accountability system for ELs’ language acquisition progress would signal to teachers and principals that these students should be an area of focus. A district-level EL accountability system would ensure that these students don’t vanish when they happen to enroll in communities with smaller numbers of ELs.
The primary challenge with building multi-tiered accountability systems (whether they’re for ELs or not) is to ensure that they signal aligned incentives to leaders at all levels. The consequences of district-level accountability systems almost inevitably involve changes to school-level practices, and vice versa. If districts commonly trigger accountability provisions related to ELs’ progress while their schools do not, this will send a confusing message to local leaders and make it difficult for them to generate genuine improvement plans in response.
One way to get around this is to use district-level EL accountability as a sort of failsafe. That is, states could decide to calculate ELs’ progress at the district level for schools with too-few ELs to meet the n-size cutoff. Though ESSA doesn’t require it, South Dakota is taking that approach.
Preparing for Disappointment?
All states have already drafted their plans for incorporating English acquisition into school accountability, but we’re still in the early days of ESSA implementation. Advocates who expected ESSA to make ELs a greater priority in school accountability considerations should be prepared to verify whether all states’ accountability systems are living up to their hopes. Given the relative rarity of using multiple years of data to meet state n-sizes (like Washington state’s) or creative strategies for incorporating English language acquisition into overall ratings (like Louisiana’s), there are real reasons for doubt.
Conor P. Williams is a senior fellow at The Century Foundation, and an expert on urban education reform, English learner students, children of immigrants, early education programs, and school choice systems. He is also a founding partner with the Children’s Equity Project.
Will Shifting English Learning Accountability to Schools Work?
Now that all fifty states and the District of Columbia have received formal federal approval of their Every Student Succeeds Act (ESSA) plans, the work of implementing the 2015 law—the United States’ primary K–12 education investment—is fully underway. As that work continues, however, advocates for educational equity should keep close eye on ways that ESSA’s structure creates new, unforeseen challenges for historically underserved students. This is particularly true as far as English learners (ELs) are concerned, given that ESSA made significant changes to how schools are held accountable for these students’ performance. Specifically, there is some evidence that ESSA’s shift of EL accountability from the district to the school level could make these students less of a priority for many schools.
Replacing No Child Left Behind
Like most of ESSA’s big changes, the law’s EL accountability reforms are a response to troubles with No Child Left Behind (NCLB), the 2001 law it replaced. Of course, these problems weren’t intentionally caused. No Child Left Behind was initially heralded as an advance for English learners in U.S. schools. Its Title III provisions provided new federal funding dedicated to serving ELs. The structure was intuitive: ELs had long been marginalized by—and in—U.S. schools. So Title III provided dedicated funding and focused accountability that would force districts to prioritize these children’s linguistic and academic development. Districts receiving Title III dollars were required to meet Annual Measurable Achievement Objectives (AMAOs): if a district’s ELs showed low academic and linguistic progress over a number of years, the district would face a range of state interventions designed to improve how it served those students.
This was a new policy experiment—ELs had never before had a stream of federal formula funding—and its limits soon became evident. While Title III was funded between $650 million and $750 million most years, it turned out that, when spread out across the country, this was hardly sufficient to focus local education leaders on ELs’ needs, particularly as the number of ELs in U.S. schools grew by approximately one million students. A national study of Title III implementation found some hesitation by state education officials when it came to imposing consequences on districts where ELs were chronically struggling. While Title III dollars and accountability put EL students on the map for schools and districts, many advocates felt that they were inadequate to making ELs’ development a top concern.
ESSA’s Solution…and a Possible New Problem
The Every Student Succeeds Act upends this status quo with an eye to addressing Title III’s shortcomings. It maintains Title III’s funding structure, but shifts the federal government’s primary EL accountability mechanisms into Title I, its primary K–12 funding stream. Title I funds support educational equity in schools serving large percentages of low-income students. Congress has funded it at over $15 billion annually in recent years. Moreover, Title I included NCLB’s signature provision: the goal that all students be proficient in reading and math by 2014, with consequences for schools and districts receiving Title I funds that fell short of annual progress toward that goal.
EL advocates argued that making ELs’ English acquisition a central part of the federal government’s largest stream of K–12 funding would lead more schools and educators to attend to ELs’ needs by including accountability checks for ELs’ progress in the government’s primary accountability system for monitoring the progress of all students. They convinced Congress: ESSA mandates that schools serving ELs will now be rated using a single system that takes into account traditional measures, like achievement in reading and math, alongside ELs’ annual progress learning English. The idea, as I noted at the time, was that ELs would get more attention if accountability for their performance was attached to that much larger funding stream. These funds—and the attached accountability systems—are too big for most schools to ignore.
That last bit is critical: ESSA’s Title I accountability requirements are applied solely at the school level, unlike the old Title III district-level accountability (importantly, NCLB’s Title I included both school- and district-level accountability requirements, but federal lawmakers removed the latter in ESSA). This poses a challenge for ESSA’s theory of action on EL accountability. While school-level accountability focuses pressure on the decision makers closest to these students, it could also—paradoxically—make ELs less of a priority for them.
This is because federal law requires states to set minimum student subgroup sizes for accountability calculations—the number of students enrolled in a school that is necessary in order to include a particular group in school ratings. These “n-sizes” are designed to support valid and reliable data collection. The smaller the size of a group of students, the more likely that their achievement numbers will fluctuate. N-size rules aim to stabilize data to be more consistent over time—especially when those data have accountability stakes attached.
N-sizes also protect students’ privacy. This should be intuitive: if there are just three or four ELs in a particular school, any sharing of that school’s EL achievement data in public accountability systems could run close to publicly revealing information about an individual student’s progress. So states also set a minimum n-size to protect students’ privacy—most are between ten and twenty-five students.
Here’s the key: If a school has fewer students in a particular subgroup than the state’s minimum n-size, that subgroup is left out of accountability calculations.
That’s why ESSA’s new approach to EL accountability could pose problems for EL advocates’ theory of action. Say, for instance, that a state sets its minimum n-size for subgroups at ten students. If an accountability system is calculated at the district level, that means that accountability provisions covering ELs’ academic and language acquisition progress only kick in when there are at least ten ELs enrolled in the district. If an accountability system is calculated only at the school level, however, that means that the accountability provisions only kick in when there are at least ten ELs enrolled in a particular school.
This isn’t a brand-new concern. Back in 2010, a study of national Title III implementation under NCLB suggested that n-size differences might have contributed to disparities in which districts were targeted for which kinds of improvement. Specifically, it noted that “districts with small populations may not have had enough ELs to qualify for a Title I accountability rating for the EL subgroup but did have enough ELs to qualify for a Title III accountability rating.” If n-sizes limited the inclusion of ELs at the district level under NCLB, it is almost guaranteed that n-sizes will limit the inclusion of ELs at the school level under ESSA—and compound the issue by drawing inequitable public attention to the problem of ELs getting inadequate educational opportunities.
Sign up for updates.
How Serious Is This?
So: in an effort to raise ELs’ profiles by moving federal EL accountability into Title I’s school-level accountability calculations, did advocates inadvertently reduce their prominence?
Let’s take a look, using a few state examples. Rough calculations using Louisiana’s publicly available district enrollment data from October 2013 suggest that over 99 percent of the state’s ELs were in districts with at least ten ELs. Given that Louisiana sets its minimum student subgroup size for accountability purposes (“n-size”) at ten students, those students would have been covered by No Child Left Behind’s Title III accountability provisions.
But if we apply the new ESSA accountability structure to this (admittedly old) dataset, things look different. According to the state’s school enrollment data from October 2013, around 85 percent of the state’s ELs attended schools with more than ten ELs. Under Louisiana’s ESSA plan, those schools would have their ELs’ progress towards English language proficiency counted in their overall accountability calculations.
In simpler terms: the new school-level EL accountability calculation would have removed over 2,000 ELs (15 percent of the state’s ELs) from Louisiana’s school accountability system that year.
Note, again, that this is an artificial scenario: in 2013, ESSA’s design and passage were still two years in the future (which is why I picked those data). To confirm the persistence of the problem, I reran the analysis with Louisiana’s 2017 numbers. That year, ESSA’s school-level EL accountability would have covered about 90 percent of the state’s ELs while NCLB’s old district-level accountability system would have covered over 99 percent. In 2017, at least 6,000 of Louisiana’s ELs were enrolled in schools that did not meet the state’s n-size—in other words, that served fewer than ten ELs (Note: there were many more ELs enrolled in Louisiana in 2017 than in 2013). Indeed, in 2017, two-thirds of Louisiana’s schools did not meet the n-size for EL enrollment.
This rudimentary analysis illustrates a real problem: moving EL accountability exclusively to the school level could make it more likely that ELs’ progress towards acquiring English language proficiency isn’t counted in schools’ accountability calculations.
Louisiana’s scenario is not unique. I ran the same analysis using Minnesota’s publicly available enrollment data (also from the 2013–14 school year). Minnesota set its minimum n-size for subgroups at twenty students; almost 99 percent of the state’s ELs were enrolled in districts with at least twenty ELs enrolled. By contrast, 95 percent of the state’s ELs are enrolled in schools with at least twenty ELs. In other words, the new approach under ESSA reduces the number of ELs included in school accountability ratings by over 3,000 children. In 2016–17, the proportions were almost identical.
What to Do?
None of this means that school-level EL accountability is fatally flawed. Indeed, advocates may be correct that those ELs counted under the ESSA’s approach to EL accountability will get significantly more attention than they would have under Title III’s district-level accountability system—and that benefit may be worth the cost of including fewer ELs in the accountability systems.
Fortunately, there are ways that states can maintain the focus of school-level EL accountability while navigating the challenge of overseeing schools with small numbers of ELs:
1. Calculate subgroup performance across multiple years.
The state of Washington’s ESSA plan illustrates one possible solution to the issue. The state set its ESSA n-size at twenty students. While developing its ESSA plan, it ran a series of analyses to see how different n-size policies would determine how many ELs would be included in the state’s accountability system. It published the results in the following table.
Look at the bottom row of the table. Just 63.5 percent of Washington’s ELs are enrolled in schools serving twenty or more ELs. By contrast, 82.8 percent of Washington’s ELs attend schools with ten or more ELs.
In response, Washington policymakers decided to set their n-size at twenty students, but allowed it to be calculated across three years. That is, if a school has 11 ELs one year, 14 ELs the next, and 10 ELs the third year, this would add up to a total of 35 data points, which would meet the state’s n-size (calculated across three years). The state found that this would include nearly 89 percent of the state’s ELs (see the cell in the bottom left corner of the table).
This may not reach the same degree of coverage as the state’s previous district-level EL accountability system, but it expands its range somewhat. Notably, a recent Alliance for Excellent Education analysis found that just nine states proposed to combine several years of data to ensure that more schools meet n-size rules for ESSA accountability.
2. Incorporate ELs’ progress towards learning English into other academic indicators.
In the ESSA plan it submitted to the U.S. Department of Education, Louisiana worried that its “minimum N size would cause the majority [of] schools to be excluded from EL progress monitoring.”
ESSA requires states to measure and rate schools on a range of metrics, not just on ELs’ progress acquiring English. Louisiana converts each of these elements—student growth and proficiency on academic assessments, high school graduation rates, and so forth—into points, so they can be combined into a single, overall accountability score for the school.
The state realized that this system gave it an option for addressing the n-size problem for school-level accountability. Despite setting its n-size at ten, Louisiana still found that over half of its schools wouldn’t meet that threshold and be covered by EL accountability. So it decided to convert ELs’ progress learning English into “points” that it could factor into their schools’ overall accountability scores. ELs attending schools with fewer than ten ELs wouldn’t have their English acquisition progress published, but it would still be counted towards the school’s overall accountability rating. In addition, ELs’ progress towards English proficiency counts more as the number of ELs in a school increases. That English acquisition metric’s share of a school’s score goes up as the school’s share of ELs increases.
In other words, every EL in Louisiana schools would be counted towards the school’s overall score. This sends exactly the signal that advocates hoped for: ELs’ English acquisition should be a priority for all schools.
3. Reinstate district-level accountability on top of school-level accountability.
It’s easy to think of new policies solely as alternatives to old ones. That is, if NCLB’s Title III EL accountability system focused on districts, and ESSA’s EL accountability provisions focus on schools, it can seem as though we have to choose. But there’s no substantive reason why U.S. public education can’t hold both schools and districts accountable for the same activity.
The advantages of a combinatory approach are obvious: a robust school-level accountability system for ELs’ language acquisition progress would signal to teachers and principals that these students should be an area of focus. A district-level EL accountability system would ensure that these students don’t vanish when they happen to enroll in communities with smaller numbers of ELs.
The primary challenge with building multi-tiered accountability systems (whether they’re for ELs or not) is to ensure that they signal aligned incentives to leaders at all levels. The consequences of district-level accountability systems almost inevitably involve changes to school-level practices, and vice versa. If districts commonly trigger accountability provisions related to ELs’ progress while their schools do not, this will send a confusing message to local leaders and make it difficult for them to generate genuine improvement plans in response.
One way to get around this is to use district-level EL accountability as a sort of failsafe. That is, states could decide to calculate ELs’ progress at the district level for schools with too-few ELs to meet the n-size cutoff. Though ESSA doesn’t require it, South Dakota is taking that approach.
Preparing for Disappointment?
All states have already drafted their plans for incorporating English acquisition into school accountability, but we’re still in the early days of ESSA implementation. Advocates who expected ESSA to make ELs a greater priority in school accountability considerations should be prepared to verify whether all states’ accountability systems are living up to their hopes. Given the relative rarity of using multiple years of data to meet state n-sizes (like Washington state’s) or creative strategies for incorporating English language acquisition into overall ratings (like Louisiana’s), there are real reasons for doubt.
Tags: ESSA, English as a second language