Prepared by Michael Heim

Prepared for Hawai‘i School Leadership Academy

Purpose
School leaders are increasingly “coming under the gun” of accountability. The purpose of this monograph is to help school leaders think about, understand, and respond thoughtfully and effectively to the increasing demands for accountability in education. Readers will acquire a comprehensive and rather sophisticated set of concepts and insights into accountability that will help them in working with staff, in building collaborative relationships with others within the Department and with external partners, and in contending with critics. The school principal, as the primary leader and chief executive officer of the school, bears the brunt of the responsibility to ensure that demands for school accountability—whether externally or internally generated—are adequately met. This monograph is not a “how to” manual. But the practical “how to’s” of accountability, as devised and practiced by school leaders, should derive from a sound conceptual foundation.

Accountability: Muddled Meanings, Increased Importance
Accountability in education, especially in the current context of multiple reforms and restructuring, is a rather muddled concept. One needs only listen to snippets of the current educational reform dialogue to realize that “accountability” has many meanings for political leaders, education officials, teachers, parents, community and business leaders, and the general public. Sometimes, accountability is used synonymously with “responsibility.” Other
times, the term appears to refer to reporting to those with oversight authority or, more globally, to the general public; or to demonstrating compliance with established laws, rules, regulations, or standards; or to distributing rewards and sanctions tied to results.

The need to be clear about what accountability means has never been more compelling. Currently, the Hawai‘i Department of Education (DOE) is developing an extremely ambitious strategic plan for a Comprehensive Assessment and Accountability System (CAAS). The system will integrate information from student assessment, personnel evaluation, school evaluation, and system evaluation. The need for CAAS derives, in part, from accountability concerns related to earlier reforms that began redistributing operational authority to schools. School/Community-Based Management (SCBM), initiated in 1989-90, is now almost commonplace: 82 percent of Hawai‘i’s public schools have submitted Letters of Intent, and 44
percent of all schools have Board-approved Proposals to Implement. Lump-sum budgeting
flexibility, under which schools can, for example, “sell” a teaching position in order to “buy” computer hardware, was initiated in 1993-94 among all schools. Board-approved state and district office restructuring plans, to be implemented beginning in the 1996-97 school year, will result in a major reorganization of the “bureaucracy.” Due to the State’s recent revenue shortfalls and the Department’s reduced budget, the restructuring will be accompanied
by significant downsizing. Consequently, continued decentralization of authority to schools
seems likely, as does an accompanying transfer of accountability.

In addition, GOALS 2000, the consolidation of federal programs under the Improving America’s Schools Act (IASA), and the Hawai‘i Content and Performance Standards (adopted by the Board of Education in October 1994), have spurred systemic and standards-based reforms focused on expanding challenging learning opportunities for all students. These reform efforts will require more comprehensive evidence of student success and the development of alternative assessment methods. The focus on student achievement, on the extent to which standards were met, can be expected to trigger related questions about the effectiveness of instructional programs, about the allocation and adequacy of instructional resources, and even about the competency of instructional personnel. Again, increased accountability seems inevitable and should be expected to follow the decentralization of authority to the school level.

Literature on Accountability in Education
A complete description of the literature review search and selection procedures used, limitations, and qualitative analysis conducted are given in the full report (Heim, 1995) on which this monograph is partially based. The professional literature in educational research and evaluation has apparently assumed that “accountability is an intuitively clear notion, ...” which as Scriven (no date, page 2) pointed out, “may not be so.” Only a relatively small set
of papers and reports were found that focused primarily on educational accountability. Much of the relevant literature—like discussions about accountability among policymakers, educators, and the public—occurs in relation to something else. Substantial writings on accountability were most often found in conjunction with the following topics: school evaluation models, educational restructuring or reform, educational indicator systems and school report cards, and school review or audit models.

Accountability, What Is It?
Accountability is multi-faceted: it involves responsibility, authority, evaluation and control. So
how might “accountability” in education be defined? Explicit definitions of accountability in
the literature were infrequent. The following “working” definition of accountability is proposed
(Heim, 1995):

Accountability is the responsibility that goes with the authority to do something. The responsibility is to use authority justifiably and credibly.

Accountability, then, is a form of responsibility. It involves at least two parties and a mutually acknowledged relationship between them. That relationship involves a delegation of authority to take some action, from one party to another. (Where no delegation of authority occurs, there should be no expectation for accountability.) That authority, however, is delegated conditionally, at minimum, upon demonstrably credible performance. Although one may hope for ideal performance, it is credible performance (i.e., at least as good as might be reasonably expected under the relevant conditions) that should be deemed sufficient. Control is exercised via the delegation of authority, which may be continued or may be withheld, conditional on credible performance.

Some of the confusion with adequately conceptualizing accountability is related to its fluid and
pervasive nature. For instance, accountability may be directed toward either process (how something was done) or outcomes (what results were accomplished). If one has been delegated the authority to engage in some activity, then one is responsible, at the least, for conducting the activity “properly”— that is, in accord with prevailing expectations that guide how the activity should be conducted. This might be termed procedural accountability.
Procedural accountability appears to encompass the definitions of “accountability” proposed by Brown (1990), Darling-Hammond (1992), McDonnell (1990), and Scriven (no date).

Accountability may extend further to include responsibility for the consequences or results of
one’s actions—whether positive or negative, and whether intended or not. This focus might be
termed consequential accountability. It includes the ERIC Thesaurus definition (1994).
Consequential accountability is implied by those who advocate that education should be “results-driven.”

Given the establishment of both national and Hawai‘i education goals a few years ago, followed by the more recent adoption by the Board of Education of the Hawai‘i Content and Performance Standards, it appears that public education in Hawai‘i, like that in many other school districts and states, is moving rapidly into standards-based reforms in curriculum, instruction, and student assessment. Note the strong consequential accountability
orientation of standards-based reforms. To the extent that current school and school system
instructional accountability mechanisms focus on procedural rather than consequential accountability, such mechanisms will be inadequate to assess, track, and evaluate standards-based reforms.

It should be noted that almost 20 years ago, the DOE had a comprehensive plan for a systemwide curriculum management system called the Foundation Program Assessment and Improvement System (FPAIS) which contained (page 5) the following definition for accountability: “Determining and accepting responsibility; disclosing results” (1977). Like most definitions found today in the national literature, this one is incomplete. While the FPAIS definition contains one of the key elements, “responsibility,” it lacks another that is essential. The evaluative nature of accountability, i.e., using authority justifiably and credibly, is an essential characteristic. The evaluative dimension is what distinguishes accountability from reporting.

LeMahieu and Lesley (1994) provide an insightful distinction between accounting and accountability. While “accounting” is prerequisite for “accountability,” it is not sufficient. The key difference is that accountability must be embedded in what they term a “process of use” (page 8). Put negatively, if public accounting information is not embedded in a process of use, such that the information can subsequently foster change, modification, or revision of current practice or policy, then one has accounting but not accountability. Along the same lines, Darling-Hammond and Snyder (1992, page 14), say that an accountable school’s operations should “provide internal self-correctives in the system to identify, diagnose, and change courses of action that are harmful or ineffective.” Accounting, then, is primarily descriptive, whereas accountability is essentially evaluative.

All of Hawai‘i’s public secondary schools, and a small but growing number of its elementary schools, engage in the Western Association of Schools and Colleges (WASC) accreditation process. Imagine if the accreditation process stopped, literally, following the production of a school’s self-study report. One would then have accounting (reporting), but not accountability. An experimental accreditation form, Focus on Learning, is now being piloted in Hawai‘i. The pilot form puts much greater emphasis on developing follow-up action plans in response to accreditation recommendations, and on implementing and assessing progress on those action plans. Given this “process of use,” school accreditation should become a more powerful basis for local school improvement, public accounting, and accountability.

School leaders are all too familiar with the dilemmas posed by conflicting rules and expectations. The kinds of expectations (e.g., bureaucratic, legal, political) that pertain to procedural accountability, especially, are well established. The accountability and accountability-related literatures contain fairly extensive discussions about what are variously referred to as “types,” “strategies,” “mechanisms,” and even “models” (a misnomer) of accountability. The different kinds of expectations, however, might be more aptly characterized as sources of accountability expectations. The multiple sources of accountability expectations contribute to the “fluid” and sometimes confusing nature of accountability encountered in practice.

The main categories of these expectations are bureaucratic, legal, professional, political, and market-based. As the labels imply, these sources of accountability expectations derive from different philosophical bases, traditions, and settings. A brief summary of each of the five sources of accountability expectations follows.

Summary of Sources of Accountability Expectations by Type
Bureaucratic: Uses hierarchic structure and authoritative superior-subordinate
relationships to enforce compliance with rules and regulations
Values promoted: equitable resource allocation, equal access, planned management, uniform/standardized operations
Major weaknesses: unresponsive to individual client needs; minimizes professional autonomy and creativity of personnel

Legal: Uses statutes to direct compliance and use of suits or injunctions to obtain redress for violations
Values promoted: establishment and enforcement of legal rights; maintenance of rights via a formal avenue of complaint
Major weaknesses: costs of monitoring compliance; reliance on punishment to induce compliance; adversarial process

Professional: Uses review by professional peers using the standard of “accepted practice” within the profession
Values promoted: professional autonomy to provide services to best meet client needs; responsiveness to variation in client needs
Major weaknesses: difficult and costly to establish the elements for developing and
maintaining a professional culture

Political: Uses the processes of democratic control (elections, political action, public opinion) to influence and constrain the use of authority by elected officials
Values promoted: democratic control; responsiveness to constituents; inclusiveness
Major weaknesses: public expectations may be vague, unclear; unwieldy in diverse, pluralistic communities

Market-Based: Uses choice of providers within a (regulated) market to obtain best services and induce quality improvement among providers
Values promoted: consumer rights; responsiveness to client preferences/needs; competition among providers
Major weaknesses: no assurance of public’s equal access to services of comparable quality; providers are likely to be responsive only to their particular clients

The five sources of accountability expectations delineate very different bases by which justification for the use of delegated authority is viewed and defined. With bureaucratic expectations, for instance, justification is often given in terms of acting in accord with established rules and regulations.

Or, for the legal source, justification often would be couched in terms of acting in accord with applicable statute or court order. Again, this points out the fluid and complex nature of accountability. As any education official or school administrator is well aware, all five sources of accountability expectations, sometimes conflicting, are simultaneously active.

A simple, but rather powerful organizing concept is the so-called basic accountability question,
“Who is responsible for what to whom?” It can be instructive to view the question in the following format (along with a few examples):

The “Who is Responsible?” and “To Whom?” components contain numerous accountability
providers and recipients: policymakers, funding agents, government agencies, education
officials, local school governance bodies, school staff, parents, students, the general public, community organizations, and special interest groups. Most of these could be detailed-out further. For instance, state-level education policymakers include the Hawai‘i State Board of Education, Legislature, and Governor.

When viewed this way, it becomes apparent that there is an internal-external dimension to
accountability. That is, some accountability relationships occur among providers and recipients
located within the same organization (internal), while other accountability relationships involve
recipients of accountability located outside the organization (external).

In addition, a given provider (e.g., education official or school administrator) is typically
engaged in multiple accountability relationships, both internal and external. The prevalence of multiple accountability demands partially explains the origin of the problem of overload and fragmented accountability mandates. Particularly for external accountability, fragmentation tends to occur because the various “ToWhom?” recipients are often jurisdictionally, organizationally, and/or administratively independent. Consequently, the mandates generated are, fundamentally, uncoordinated and disconnected from each other.

The basic accountability question is useful also for assessing the scope of accountability in an organization. The “For What?” component in the Hawai‘i DOE can be described in terms of the following broad categories:

• funds
• personnel
• facilities
• protection of primary clients (e.g., safety, due process)
• direct services
• support services
• other administrative services
• organizational management

Closer to the operational level of detail, any one of these areas is complex and enormously
detailed. Personnel, for example, might be further described in terms of areas such as recruitment, selection, classification and certification, compensation, personnel evaluation, personnel development, collective bargaining, and affirmative action. Any one of these sub-areas, in turn, could be further described in terms of applicable laws (federal and state), policies, regulations, department directives, guidelines, standard operating procedures and
forms.

The overall scope of accountability in the Department of Education is enormous. It can only be concluded that vocal critics who complain that education “lacks accountability” either don’t know what they’re talking about, or know what they’re talking about—for a specific area in which sufficient accountability may be lacking (e.g., “accountability” for student outcomes). The latter topic is covered in a subsequent section.

The following diagram serves to integrate various but incomplete accountability frameworks
found in the literature with the definition of accountability used herein. The diagram appears to
contain all the components required of a conceptual model for accountability.

Conceptual Accountability Model

* Standards for procedural accountability, in particular, include the accountability expectations from bureaucratic, legal, professional, political, and market-based sources.

The conceptual accountability model contains the three elements of the basic accountability question, “Who is responsible for what to whom?” As noted earlier, the focus of accountability (“For what?”) may be on process or outcomes. The controlling party or recipient of accountability (“To whom?”), may be internal or external to the provider (“Who is responsible?”). The remaining three components of the model are needed to complete
the evaluative and control aspects of accountability.

Consideration of “relevant limiting conditions” is a necessary adjunct to the “evaluation” component. Judging whether performance was “credible” means evaluating whether the performance was at least as good as might be expected, given the relevant conditions. Lack of consideration of conditions affecting performance, whether limiting or enhancing, is unfair.

The sources of accountability expectations previously discussed (e.g., bureaucratic, legal, professional) are subsumed in the “standards” sub-component of the “evaluation” block. As previously mentioned, these sources of accountability expectations sometimes occur simultaneously and may conflict. Seen in the abstract form of the model, it appears that the same performance data could generate different findings, and even different implications for
follow-up, if different standards (types of accountability expectations) are applied.

Before turning to school/classroom accountability and accountability for student outcomes, two
difficult problems will be brief ly explored: the worsening problem of “accountability overload”
and the long-standing problem of defining appropriate consequences. What follows should be considered by policymakers as well as practitioners.

“Accountability overload” refers to a condition in which the accumulation of accounting and
accountability demands has become excessive; i.e., no net value is being added to the product produced or the service provided, or, worse, value may be actually reduced due to the diversion of staff time and effort. The steady accretion and layering of accounting and accountability mandates is rarely accompanied by the removal of earlier requirements.
Several steps could be taken, either singly or collectively, to better control and even reduce “accountability overload.” To succeed, the suggestions that follow would require concerted effort and commitment on the part of policymakers and practitioners.

(1) Accountability, as defined herein, includes a “context for use” that serves to
periodically evaluate the benefits of having conditionally delegated authority, and
may include other consequences, positive or negative, as well. It is at that point in
the accountability cycle where control is exercised, and, most importantly, where
the greatest potential for improvements can be realized. Where possible, accounting
activities should be (a) deleted, (b) reduced in scope, or (c) merged into accountability activities.

(2) Where possible, accounting and accountability activities directed toward processes should be deleted or reduced in scope. Desired outcomes cannot be
obtained, except by chance alone, without clearly focused effort—individual and
organizational—on achieving the desired outcomes. Consequential accountability
for outcomes should be emphasized. Consider the following hypothetical
example. Suppose that 1 percent of Organization A’s total resources are consumed
by accounting and accountability activities that are mostly process focused.
In contrast, suppose that Organization B spends 1 percent of its resources on
accounting and accountability directed mostly toward its major desired outcomes.It is not difficult to imagine which of the two organizations is most likely to be
using the remaining 99 percent of its resources more effectively and efficiently.

Defining appropriate consequences in the context of accountability in public education is both
complex and controversial, but also underdeveloped and in need of more thought and public dialogue. Consequences (evaluation usage) are an essential component of accountability, i.e., authority is delegated conditionally upon demonstrably credible performance. At present, though, the consequences of poor performance often seem lacking or ineffectual. Yet a cautionary note about “beefing up” consequences is warranted. The imposition of consequences, especially high-stakes ones that may impact a student’s current or future learning or work opportunities, or an adult’s career, is a serious matter. If valid information about performance is lacking, or if standards for evaluating performance are unsound or unfair, then resistance to the imposition of high-stakes consequences is an entirely rational response.

Serious accountability will require great clarity delineating “Who is responsible for what to
whom?,” careful consideration of conditions that may limit actual performance, valid evaluation of performance (including relevant expectations or standards), and enactment of reasonable, fair consequences. In sum, serious accountability will require disciplined and careful effort.

School/Classroom Accountability for Student Outcomes
Increasingly, as the Hawai‘i DOE decentralizes more operational authority to schools and concurrently downsizes its state and district offices, the locus for accountability will shift to local schools at an accelerated rate. Parents and community members will increasingly become key players in school affairs, and school leaders must be able to demonstrate accountability. Traditionally, attending to the demands of internal and external accounting and accountability in the Department was mostly district and state office work. Of the various accountability
demands, most troublesome will be accountability for student outcomes.

While considerable control often can be exercised over how a particular activity is conducted, it
is possible that, even though an activity has been conducted in a completely acceptable manner, the final results or outcomes may be unsatisfactory. Typically, all the necessary conditions that will lead to an expected outcome cannot be controlled. Student learning, for example, is known to be influenced by many factors, some of which are within the authority of school personnel to control, and others which are not. Student learning is co-produced. The primary producers are teachers, students themselves, and parents.

Although the difficulty of the educational task may vary greatly from school to school, from classroom to classroom, and from student to student, and although all the factors contributing to successful learning may not be fully controlled or even fully understood, the most basic compact between the schools and the public is that the public school and its teachers must make a difference in the lives of children and youth. School accountability for student outcomes, then, must highlight the ways and extent to which the school has contributed to making a difference. Information that shows students’ status in terms of performance standards at a point in time is useful, but information that shows growth or improvement over time is essential.

The Hawai‘i Content and Performance Standards represents, essentially, a first step in
negotiating a preliminary “for what” component in the area of accountability for student outcomes. Much more needs to be done: aligning curriculum, classroom instruction, and student assessment practices with the standards. Educators are responsible for accomplishing these tasks. Educators are also responsible for truly involving others so that the “co-production” of learning can be successful. Attention must be given throughout these efforts to
thoughtfully cultivate an ongoing “process of negotiation” with parents and community members. Accountability in public education must be developed through a process of negotiation among all participants and stakeholders. Accountability between students and teachers, teachers and parents, and students and parents, cannot exist without mutually acknowledged relationships and responsibilities among the parties involved.

The Success Compact, “Reading, Writing, and Relating — Every Student, Every Time,” therefore
is a literacy-focused instructional reform initiative of the Hawai‘i DOE that emphasizes school-by-school change, with teachers collaborating together as professionals to improve and to articulate their classroom instruction, thereby building a community of learners. Total quality concepts, such as “What works best for the learner?,” are central to the Success Compact. As noted by Superintendent Aizawa (1994):

“Like a musical theme, What works best? is repeated over and over again by students, teachers, and parents, throughout all grades, throughout all subjects, throughout all learning activities, until it becomes ingrained as part of the school’s culture or “the way we do things around here.” (page 6)

It might be observed that cultivating a “process of negotiation” among teachers, students, and parents for accountability for learning is compatible with, and should be subsumed within, the larger effort of building a community of learners. In addition, the total quality theme, “What works best?,” needs to be linked to the performance standards, instructional practice, student assessment, staff development, and, eventually, staff and school evaluation.

A conceptual model for school accountability focused on classroom instruction is shown in the following diagram. Both learning processes and outcomes are shown as embedded in a common
context. The various contextual elements affect school/classroom processes and outcomes pervasively and in complex ways. Generally, the model is meant to suggest that both process and context determine outcomes.

All the components interact dynamically. Said differently, the knowledge of the direction and nature of causal relationships involved in classroom learning is fragmentary at best. Therefore, outcomes may also influence process and context. In the very long run, shouldn’t the outcomes of schooling contribute to altering the context of individuals, future families, and the community as a whole? Isn’t the reason why the public supports public education—to improve the “context” for future generations?

The model can serve as a useful organizer for thinking through and trouble-shooting key aspects of the question, “What works best?” In doing so, school leaders working with staff and interested school-community members will necessarily begin to operationalize an accountability “process of use” focused on the core “business” of the school—effective classroom instruction for improved student learning. For instance, in the situation where student academic achievement is viewed as inadequate, then learning expectations (which include, but are not necessarily limited to, the Hawai‘i Content and Performance Standards), instructional practices and opportunity to learn, student effort or motivation, and classroom social climate—each shown in the diagram as main elements of instructional “process”—could comprise the key targets for review, and, if warranted, for subsequent improvement. Related context elements, especially family support, while not under the school’s direct control, can be influenced by the school staff to some degree. It is important to keep in mind that student learning is co-produced, and that even overused catch-phrases, such as “parents as partners in education,” do have a real and powerful meaning.

School Accountability Focused on Classroom Instruction

Although student assessment is not explicitly shown in the model, it is implied by the “outcomes” block. As noted earlier in this monograph, a strategic plan for a Comprehensive Assessment and Accountability System (CAAS) is under development. A substantial part of that plan will address current and emerging student assessment needs. Most likely, a two-level student assessment system will be proposed: operationally separate school and state assessment subsystems, both linked, though, to the Hawai‘i Content and Performance Standards. One might envision a state assessment subsystem that is designed mainly to provide school-by-school and statewide information for school and system progress monitoring, and a parallel series of highly specific classroom assessments designed mainly for teachers’ use in assessing individual student progress. The latter would be supported by an
Assessment Center, an electronic library of assessment materials, links to other sources of assessment information, and, if funded, assessment-related technical assistance and staff development services.

The accountability model proposed earlier contains consequences. How should consequences be tied to student assessment? High-stakes consequences for students, particularly negative ones, should not be considered until questions about the technical adequacy of assessments and equity issues related to opportunity to learn have been resolved. Meanwhile, the single best accountability use of standards-based assessments about student outcomes is for improving instruction, i.e., improving what works best for the learner. (Note the arrow in
the diagram on page 22 pointing from “outcomes” to “process.”) School leaders can also play a key role in assisting teachers to internalize the view that assessment information, while nominally a “readout” on student learning, also reflects the effectiveness of the teacher’s instruction. Outcomes cannot be guaranteed. Learning is co-produced. But teachers, to be regarded as professionals, need to credibly demonstrate their use of “best practices,”
including the use of outcome information to improve that practice.

References
Aizawa, H. M. (May 1994). “Success Compact: Every Student, Every Time. Superintendent’s Vision.” Honolulu, HI: Hawai‘i State Department of Education.

Brown, P. R. (1990). Accountability in Education. Policy Briefs, No. 14. Far West Laboratory for Educational Research and Development, San Francisco, CA. [ED326949]

Darling-Hammond, L. (July 1992). Standards for Practice for Learner-Centered Schools.
National Center for Restructuring Education, Schools, and Teaching.

Darling-Hammond, L. & Snyder, J. (1992). Reframing accountability: Creating learner- centered schools. In A. Lieberman (Ed.), The Changing Contexts of Teaching (Ninety-first Yearbook of the National Society for the Study of Education), Chapter 2, pp. 11-36. Chicago, IL: University of Chicago Press.

Department of Education. (October 1977). Foundation Program Assessment and Improvement System. Honolulu, HI; Hawai‘i State Department of Education, Office of Instructional Services.

Heim, M. (February 1995). “Accountability in Education.” (unpublished paper submitted to the Hawai‘i Department of Education in partial fulfillment of professional improvement leave requirements). Honolulu, HI: Hawai‘i State Department of Education, Office of the Superintendent.

LeMahieu, P. G. & Lesley, B. A. (1994). “State Education Agencies: Partners in Reform.”
(In Jane Armstrong, Ed.) Washington, D.C.: National Governors’Association. [Cited with permission of the authors.]

McDonnell, L. M. (September 1990). “Accountability and School Restructuring: Resolving the Dilemma.” Paper prepared at the RAND Corporation for the Office of Educational Research and
Improvement, U.S. Department of Education, Washington, DC.

Scriven, M. (no date). “The Foundations of Educational Accountability.” Kalamazoo,
MI: Center for Research on Educational Accountability and Teacher Evaluation.