Beyond The Numbers

Information for this page was gleaned from: White, Stephen. (2005). Beyond the Numbers. Englewood, CO: Advanced Learning Press.

Beyond the Numbers - Summary

Data-driven decision making has become an axiom in public education over the last decade. Beyond the Numbers describes the foundational principles for data analysis and provides the strategies necessary to turn that analysis into action. Stephen White outlines a different way of looking at the data every school gathers by focusing just as much on "cause" data as "result" data. The concepts explored in the book meld perfectly with the professional learning community approach to facilitating continuous school improvement. White posits that the instructional strategies that really lift performance in the new centruy will emerge locally, will represent replication of highly effective practices, and will come from the collaborative wisdom of teachers and school leaders in our schools today. The message of "Beyond the Numbers" can be summed up in the five Rs of data analysis:

• Recognize the influence of the rearview-mirror effect on our current practices, polices & values about teaching and learning.
• Realize that the data provide opportunities that require thoughtful analysis, infusion of our own expertise and insights, and decisions that change how we practice the craft of teaching.
• Reflect on available data with other professionals, engaging the power of collaboration to examine student work, implement and monitor insightful changes, and improve student achievement.
• Respond to urgent challenges.
• Replicate practices that work to share the wealth of knowledge & expertise that exists in every school.

The Rearview-Mirror Effect

Stephen White defines the rearview-mirror effect as planning the future on the basis of events that occurred in the past, and he notes that it has four debilitating characteristics. The first harmful characteristic of the review-mirror effect is responding to a rapidly changing reality based on past events. The rearview-mioor effect fails to anticipate urgent challenges and failst o elicit fresh feedback from students, parents, and teaching about the reality they are experiencing now. A common example of the rearview-mirrow effect in action is educators waiting for instructional practices to be verified before they allow changes to be introduced.

The second debilitating characteristic of this effect is waiting for the road to reveal itself, by depending on & waiting for annual assessments. Annual assessment can be valuable for analysis, but reliance on annual assessments contributes to the review-mirror effect because it creates a situation in which any response to the data is tool little and too late. The more critical data is that which underlies more formative assessment, and the teaching and administrative dimensions that make up the antecedents to excellent performance.

The third unhelpful characteristic of the rearview-mirror effect is its focus on a single dimension of the highway; that is, a focus only on what students do. This is not the same as focusing on student achievement, through a system where decisions are calibrated and measured against their impact on student achievement. The most successful school systems understand that teacher behaviors, professional development, learning conditions, resources, teacher qualifications, curriculum alignment and development processes, assessment variety, common planning, and a host of other antecedent conditions and structures influence student achievement. Data is not just numbers, and examining only student achievement data is a recipe for frustration. In this century, schools that rely solely on student achievement data to make decisions shortchanges themselves and fail to access the expertise, wisdom, and intelligence of their faculties.

The fourth debilitating characteristic of the rearview-mirror effect is a wistful looking-back to a time when things were simpler. Sustained, breakthrough improvements in student achievement that outperform expectations will never occur when those responsible for making the improvement look backward for answers. When professionals examine student work, when they take a close look at correlations and ask why, and when antecedents of excellence that precede improved performance are revealed and institutionalized, students will surprise us and colleagues will surprise each other.

Data Analysis

Douglass Reeves recommends a feedback cycle for continuous improvement that involves: (1) formative & summative evaluations, (2) decisions informed by that feedback, and (3) actions taken on the basis o fthat feedback to make improvements and establish new initiatives. The Center for Performance assessment has created a 7-step data driven decision making process that incororates the cycle. A graphic of the DDDM process is shown below:

Antecedents of Excellence

Antecedents of excellence are strategies employed by a specific school that lead to academic achievement and the attainment of other system-wide goals. Antecedents are structures & conditions that precede, anticipate, or predict excellence in performance. Antecedents include causes, instructional strategies, administrative structures, and conditions for learning. There is a separate wiki page devoted to Antecedents of Excellence - click the link to visit it.

The Power of Collaboration

Collaboration is the platform that translates data into decisions. Stephen White writes that the incomplete nature of the data we have available is the very reason we need to add the human perspective, and it is the reason collaboration is so critical in data analysis. Collaboration also provides an opportunity for educators to benefit from the collective wisdom of professionals and, in so doing, discover how the whole really is greater than the sum of its parts. Collaboration only happens when It happens only when leaders commit to creating the systems that embed collaboration in the routine practices of the school and when they provide teachers and teams with the information and support essential to improve practice. Education must be "de-privatized" according to White, elminiating what he calls the "Lone Ranger Factor" - whereby teachers view the profession as one in which they have the obligation to independently develop their skills & content knowledge to the best of their abilities; also have the latitude to develop their own style, set their own pace, and enrich the district’s curriculum by providing in-depth units of their own choice.

Accountability

White observes that accountability is, first and foremost, taking action on the basis of what the data tells us and acting quickly on the diagnosis, rather than allowing problems to fester. Accountability is also student-centered, meaning that it relies on measures of both student achievement results and antecedents of excellence. Accountability for data analysis means that teachers provide leadership in the analysis process and that such leadership is fundamentally collaborative. White defines the concept of accountability as: (1) the authority to commit resources (to take action), (2) the responsibility to demonstrate improvement (results), and (3) permission to adjust time and opportunity (permission to subtract) so that all students achieve beyond their expectations and the expectations of adults committed to their achievement. In "Beyond the Numbers", White shifts the thinking of accountability from the traditional obsession with test performance and adequate yearly progress to an examination of classroom practices - which have a greater effect on student achievement than professional development, teacher quality, or any demographic variable, including race, income, or educational level of parents. It's an important shift for leaders to recognize, especially in the face of accountability systems developed by New York State and the federal government.

Canaries in the Coal Mine

White makes an analogy to the mining industry when he states that the rearview-mirror effect will continue to plague educators until they and their schools develop their own “canaries in the coal mine.” Schools need early warning systems that allow teachers and administrators to make adjustments quickly and respond with agility based on student needs. White lists the following "canaries" as ideas for schools to consider:

• Data teams provide a very versatile framework within which to probe and monitor student performance in ways that facilitate rapid and focused response.
• Performance assessments that rely on scoring guides as blueprints to proficiency offer a very sound approach to learning that informs teachers, students, and even parents of a student’s progress along the way.
• Performance assessments that are designed to include interim performance-task indicators, act as even more certain “canaries” that allow educators to respond to student needs.
• Routine, corrective, and instructive feedback as a work habit assists teachers to make adjustments almost immediately.
• Liberal use of self-assessment efforts constitutes yet another approach that allows the professional educator to make midcourse corrections.

Triangulation

Triangulation is an approach to data analysis which ensures that the principles of collaboration, antecedent identification and monitoring, and accountability are addressed. The approach draws extensively on various data analysis tools, but ultimately relies on the judgment of professionals to make their best decision given limited data. Triangulation examines the intersection of antecedent data, collaboration data, and accountability data. It is a method for extrapolating meaning from raw data; a means to find the critical information, see the big picture, and identify key component. When data is triangulated each point serves as a check on the other dimension, with the desired outcome from use of the various data points (and types) always being the realization of new insights that are not available from examining only one type of data or one perspective. One tool for triangulation is the Wagon Wheel - an example of which can be downloaded from the Resources section of this wiki page.

Beyond The Numbers Resources

Click the links below to access documents that Sweet Home administrators have used to build understanding around the concepts presented in Beyond the Numbers.