<%@LANGUAGE="JAVASCRIPT" CODEPAGE="1252"%> Using Standards-Led Policy to Align Assessment and Accountability Systems

Pacific Resources for Education and Learning
900 Fort Street Mall • Suite 1300 • Honolulu, HI 96813
Phone: (808) 441-1300 • Fax: (808) 441-1385
Email: askprel@prel.org • Website: www.prel.org
 

Using Standards-Led Policy to Align Assessment and Accountability Systems

by Don Burger*

 

Briefing Paper
Product # PB0206

Introduction
Education policy makers are concerned that students in the United States are not achieving at expected levels. Since 1965, the federal government alone has spent in excess of $321 billion dollars on improving education, specifically early education and reading. Still, according to the U.S. Department of Education (2002c), little progress has been made over the past 20 years.

In 1994, the Improving America’s Schools Act and Title 1 of the Elementary and Secondary Education Act required states to create clear expectations for student learning (content standards), assessments that measure those standards, and systems that held school systems accountable for student achievement. The act required that all students be tested once during each of the following grade spans: 3-5, 6-9, and 10-12. By the 1999-2000 school year, 48 states had adopted state-level tests in reading and math for one grade level in each of the grade spans (Goertz, Duffy, & Le Floch, 2001).

The most recent federal effort to produce substantial improvement in student achievement scores, No Child Left Behind (NCLB), requires states and districts to ensure that all students are performing at proficient levels in reading and math by the 2013-2014 school year. NCLB requires that each state and district have a single accountability system for all students. The states must create a system that includes a) content standards by grade level, b) assessments that are aligned with standards, c) state/district/school reporting procedures, and d) an accountability system that holds the state, districts, and schools accountable for student achievement (U.S. Department of Education, 2002a).

Current Status of State Accountability Systems and Aligned Assessments
For many years, policy makers have experimented with many different strategies in an attempt to increase student achievement. Perhaps two of the most foundational strategies are a) to create a standards-based accountability system and b) to use content standards to align and integrate curriculum, instruction, and assessment.

As evidenced by the increased accountability requirements, NCLB reflects policy makers’ frustrations. Beginning in school year 2005-2006, students must be tested in reading and math every year while in grades 3-8 and once in high school. Annual expectations have also changed from annual improvement in student scores to all students being proficient by 2013-2014. In addition, the requirement for annual yearly progress has been expanded from applying to only Title 1 schools to all schools.

Strategy One: Standards-Based Accountability Systems
Standards-based accountability systems are more than just annual end-of-year standards-based tests. These systems actually incorporate the purpose of schooling, specify what is expected to be learned and who is to be held accountable for student learning, establish annual student achievement goals and reporting requirements, and determine rewards and consequences for meeting or failing to meet expectations.

There is not complete agreement on accountability system components in all 50 states, but most agree that all systems should include certain key elements (Baker, Linn, & Koretz, 2002; U.S. Department of Education, 2002b). One of the most important components is that clear expectations are set that specify learning targets (content standards), contain data elements both teachers and administrators need to improve student learning, and include all students in the assessment process.

Accountability systems should also include assessments that accurately measure the learning targets, identify the link between content standards and assessment, consider how children with disabilities or English language learners will participate, contain multiple assessment sources (no single test decision making), include multiple forms where tests are repeated, and specify and appropriately use test data (validity).

The systems have rules and processes for setting annual academic achievement goals (annual yearly progress). These include non-academic measures of system effectiveness that indicate which measures are used (e.g., graduation rate and/or dropout rate) and how they are calculated and reported. In addition, a successful accountability system involves parents and the community, making achievement results publicly available through formats that are easy to understand. Rewards, sanctions, and supports for adults and students in the system should also be specified.

Finally, all accountability systems must provide an appeals process, incorporate margins of error in tests and non-academic calculations, assist school systems in their transition from their current accountability plan to the new plan, and provide support systems to address issues identified.

Strategy Two: Alignment With Standards
The second foundational strategy is using standards-based policy to align curriculum, instruction, and assessment with content standards. Misaligned curriculum, instruction, and assessment is one factor that leads to poor student achievement. In the past, standardized norm-referenced tests, used traditionally for accountability, have only partially aligned with curricular materials and classroom instruction. These conditions obviously result in poor test scores.

The standards-led alignment approach uses policy to align, integrate, and connect components of schools as systems (e.g., assessments, curriculum, instruction, and accountability). According to Linn & Herman (1997), standards-led alignment should use local content standards as the focal point to:

  • foster the use of multiple assessment sources and methods,
  • describe how classroom and accountability assessment relate to each other,
  • align accountability and classroom assessment with learner outcomes, and
  • ensure that teachers and administrators use appropriate forms of assessment, are skilled in interpreting data, can plan for re-teaching activities using data, and can evaluate the impact of specific programs and instructional strategies.

To begin the alignment process, Allington and Cunningham (2002) advocate a comprehensive policy review to determine where all system elements connect (e.g., curriculum, instruction, and leadership). For example, standards-led assessment policy may be linked to curriculum development, curriculum adoption, grading, and professional development (Chappuis, 2002).

Implications for No Child Left Behind Requirements
As stated earlier, the two strategies identified can be considered the foundation of an effective means of improving student achievement. This conclusion is further bolstered by a report of the Commission on Instructively Supportive Assessment (2001). This report studied the possibility of creating large-scale accountability tests that also inform classroom instruction. The report identified nine requirements for standards-based accountability systems.

Requirement 1. Prioritizing Content Standards: A state’s content standards must be prioritized to support effective instruction and assessment. Achievement standards should be high but attainable. The standards must be based on sound learning theory (Linn, 1998).

Requirement 2. Describing Content Standards: A state’s high-priority content standards must be clearly and thoroughly described so that the knowledge and skills students need to demonstrate are evident. Alignment, a key feature of standards-led assessment policy, is the degree to which the assessment accurately reflects the standard being measured (Linn & Herman, 1997). The impact of alignment is that students are tested on what they are expected to know and do. Standards-led assessment policy is different than traditional assessment policy in that curriculum, instruction, classroom assessment, and accountability assessment are all aligned and linked (Baker & Linn, 2000).

Requirement 3. Standard-by-Standard Reporting: The results of a state’s assessment of high-priority content standards need to be reported standard-by standard for each student, school, and district. NCLB requires that state assessment systems report disaggregated results.

Requirement 4. Using Multiple Measures: Standards-led assessment policy specifies multiple measures. NCLB requires states to adopt or develop annual state assessments aligned with content standards in reading and math by 2005-2006 and in science by 2007-2008. The assessment system must include multiple measures rather than rely on a single test score and provide individual student data that clearly describe the student’s achievement. The assessment should also measure higher order thinking skills and understanding (U.S. Department of Education, 2002a).

Requirement 5. Monitoring Curricular Breadth: A state must monitor the breadth of its curriculum to ensure that instructional attention is given to all content standards and subject areas, including those that are not assessed by state tests. A state must also provide educators with optional classroom assessment procedures that can measure student progress in attaining content standards not assessed by state tests. When high stakes systems are employed, instructional time is devoted to teaching what will be measured. Some researchers argue that when teachers “teach to the test,” the curriculum narrows (McNeil & Valenzuela, 2001).

Requirement 6. Creating Assessment for All Students: A state must ensure that all students have the opportunity to demonstrate their achievement of state standards; consequently, it must provide well-designed assessments appropriate for the broad range of students, with accommodations and alternative methods of assessment available for students who need them.
Standards-led assessments are different from traditional accountability assessments because they are closely linked with curriculum (what gets tested has been taught); the performance standard (how good is good enough?) is usually pre-set (the target is clear and fixed); and student-constructed responses (short answers, essays), demonstrations, or performances can address some standards better than student-selected responses (multiple choice, true/false, matching, fill-in-the-blank) (Linn & Herman, 1997).

Requirement 7. Allocating Sufficient Test-Development Time: A state must generally allow test developers a minimum of three years to produce statewide tests that satisfy the Standards for Educational and Psychological Testing and similar test-quality guidelines.
Requirement 8. Investing in Professional Development: A state must ensure that educators receive professional development focused on how to optimize children’s learning based on the results of instructionally supportive assessments.

Requirement 9. Ongoing Evaluation: A state should secure evidence that supports the ongoing improvement of its state assessments to ensure they are a) appropriate for the accountability purpose for which they are used, b) appropriate for determining whether students have attained state standards, c) appropriate for enhancing instruction, and d) not the cause of negative consequences.

Conclusion
The thrust for student achievement accountability continues to increase, while policy makers try a variety of strategies to improve student achievement. Two foundational approaches are to use policy to develop a standards-based accountability system and to align curriculum, instruction, and assessment with content standards. In addition, recent federal U.S. Department of Education legislation, NCLB, requires state and districts to develop systems and strategies that result in all students being proficient or better in locally-developed content standards. End-of-year tests alone will not produce the desired achievement gains.

Standards-based accountability systems create the framework for holding the system accountable for student achievement. Aligning system components ensures that expectations for student learning are not only taught but tested as well. Implementing both these strategies to improve student learning is essential. As school policy makers continue to ponder which of the many strategies to utilize to meet accountability obligations, we encourage implementing not either but both strategies presented here. To do less would only result in more of what has plagued education for years.

References
Allington, R. L., & Cunningham, P. M. (2002). Schools that work: Where all children read and write. Boston, MA: Allyn and Bacon.

Baker, E. L., & Linn, R. L. (Winter, 2000). Alignment: Policy goals, policy strategies, and policy outcomes. Los Angeles: The CRESST Line, UCLA.

Baker, E. L., Linn, R. L., & Koretz, D. (2002). Policy brief 5: Standards for educational accountability systems. Los Angeles: The CRESST Line, UCLA.

Chappuis, S. (2002, July). Setting district policy for productive assessment. Paper presented at the Assessment Training Institute Conference, Portland, OR.

Commission on Instructively Supportive Assessment. (2001, October). Building tests to support instruction and accountability: A guide for policy makers. Available at www.aasa.org/issues_and_insights/assessment/building_tests.pdf

Goertz, M. E., Duffy, M. C., & Le Floch, K. C. (2001). Assessment and accountability systems in the fifty states. Consortium for Policy Research in Education, Number RR-046. Philadelphia: University of Pennsylvania.

Linn, R. (1998). Standards-based accountability: Ten suggestions. Los Angeles: The CRESST Line, UCLA.

Linn, R., & Herman, J. L. (1997, February). A policy maker’s guide to standards-led assessment. Denver, CO: Education Commission of the States.

McNeil, L., & Valenzuela, A. (2001). The harmful impact of the TAAS system of testing in Texas: Beneath the accountability rhetoric. In G. Orfield & M.L. Kornhaber (Eds.), Raising standards or raising barriers? Inequity and high-stakes testing in public education (pp. 1-18). New York: Century Foundation Press.

U.S. Department of Education. (2002a). No Child Left Behind: A desktop reference. Jessup, MD: Education Publications Center.

U.S. Department of Education. (2002b). Standards-based accountability systems – Key issues brief. Available at www.ed.gov/offices/OVAE/HS/ standacctb.html

U.S. Department of Education. (2002c). Why No Child Left Behind is important to America. Available at www.nochildleftbehind.gov/next/stats/index.html

Voke, H. (2002, October). What do we know about sanctions and rewards? Available at www.ascd.org/readingroom/infobrief/issue31.html

For assistance on accountability and assessment policy, please visit the following websites:

*Dr. Don Burger is the Assessment Team Leader for the Pacific CHILD project and the Director of Pacific Assessment Systems & Services at PREL.

This product was funded by the U.S. Department of Education (U.S. ED) under the Regional Educational Laboratory program, award number ED01CO0014. The content does not necessarily reflect the views of the U.S. ED or any other agency of the U.S. government.