Posting Questions

Here's a place to pose and respond to questions while you read Beyond The Numbers or consider the ideas on our Wiki. Go to the edit page, type an exclamation mark (!) and the question topic. Hit return twice. Type two asterisks, the question and then two more asterisks. . If you want to respond, click the edit page, click underneath the question/response and then respond

Triangulation

Schmoker talks about simplifying and demystification of the improvement process. The entire chapter on triangulation seems to be the complete opposite. When do we triangulate? How much triangulation do we engage in before we "do" something?

Action should not be contingent upon triangulation. Rather we need to take action based on our inferences, hunches and best hypotheses. Triangulation helps us better understand if what we do makes a difference. Triangulation is at the heart of why we monitor. This gets to perhaps an even bigger question, "Which variables or antecedents should we try to foster first?" If we decide that non-fiction writing, the posting of learning goals, and faculty meeting activities are variables that we want to monitor as possible antecedents, how do we know if we have identified the right ones? Triangulation allows us to connect antecedent/ cause data with results data.

The Wagon Wheel seems to be a highly valuable tool in identifying correlations between cause data and results data. If no discernably different patterns exist between differences in achievement and any of the variables on the wheel, we must then ask ourselves if we have the right variables?

Ann Laudisio:

Ok - be patient as I am trying this for the first time. The triangulation process is a little overwhelming to me. I think I get the concept but hope that my understanding evolves as a result of our work and the Summer retreat. I was intrigued by the discussion about "work habits' in the chapter on accountability. My hunch ( and research shows) has always been that the action and work habits of teachers has a powerful impact on achievement and learning. Reading about the study by Wnglinsky (2002) that verifies that classroom practices and work habits are key . They are a strong antecedent. So to me the next step would be to move from student data as our main piece of data to also include the monitoring piece we have been discussing. What does effective teaching and classroom practices look like? How will we monitor them? Will we have the focus and strength to move in that direction when these issues become problematic for all of us? We've touched on these issues before - walkarounds, etc. but never really stayed with it.

Jenifer Gilson:

I look at triangulation as making sure that we, and our teams, look at all data prior to making any decisions. That is, consider: the results from monitoring the antecedents, the context in which teaching and learning occur, the structures in place that support/inhibit teams and individual teachers, the strategies being implemented in the classroom, and the assessment data. I believe that, as administrators, we need to provide structure to the data process so that teams approach analysis in a systemic way - with a beginning that leads them through activities that give rise to ideas, relies on the data to validate or invalidate hunches, and has a rubric against which they can measure their effectiveness. This whole chapter on triangulation makes it seem daunting, but it is likely that teams are already leaning toward this type of exploration, they just do not have a framework within which to structure it. Educational data triangulation is about time, space, and people...do not just look at the numbers, go Beyond the Numbers.

Tony:

Think of the three tier accountability system. Tier 1 refers to the results data- student achievement results- what the kids are doing. Tier II refers to data that provides information on the causes of student learning- antecedents- what the adults are doing. Tier III asks us to tell the stories Beyond the Numbers by looking at the connections between the causes-the antecedents- and the effects- student learning and the indicators of that learning that we find in student achievement results. We need to triangulate the varied data points-both effect and cause- to begin to determine if what we are doing really makes a difference. It is only when we begin to systemically collect data on those factors we thnk are most important will we begin to identify some correlations and be able to make some informed decisions about the impact of our efforts. Take Thoughtful Ed for example. Right now- do we have any real way of knowing if our efforts have made any difference? My head tells me no. Until we actually track and monitor the breadth and depth of its implementation and then lay it alongside student achievement over time will be to make any real judgments about its impact. Consider our thinking about the impact of effective collaborative teamwork. Is there a real connection between student learning and effective teamwork? We have to really determine what student learning indicators are valid measures and find a way to quantify collaboration. Once done, we can create a two-point triangulation. IF we can quantify Thoughtful Ed, now we can have a three-way triangulation? Do effective collaborative teams whose members have taken TE training achieve better results than those teachers on effective teams but not in TE? If we don't triangulate, will we ever really know?

Joyce Brace:

I don't have my book here but my concern is about spending a great deal of time engaged in some very sophisticated analysis of hunches that might even appear to be correlated to results. I haven't read Results Now yet but I suspect that Schmoker would advise caution. What about the KISS philosophy. Isn't much of what is in this book a contradiction of this? I will add more when I get back to my book at home tonight.

In Beyond the Numbers, there is a good discussion of antecedents on page 28. I particularly like the nonexamples: Antecedents are not measures of students performance or actions of students. ...but they can be measures of adult behaviors or numbers of teachers with training and certification in specific methods such as...

We can measure those things but what good is training if the teachers never implement the strategies in the classroom?White even makes that point on page 47-"Even the most effective instructional strategy , however, will not produce breakthrough results if it is not employed as intended....When student growth fails to equal that achieved by the originators, the proven strategy is often blamed or criticized rather that the degree of quality implementation. I see "learning logs" in some classes but weak student prompts and uses. Does this mean the concept of the learning log is flawed or that the teachers never really understood it or received feedback on its use?

Triangulation is too complicated! Two-way intersection, Three-way intersection, Four-way intersection....what?

Joyce-learning is too complicated!! There are many variables that come to play in the learning environment. It is not enough to simply say "hooray" or bemoan the new set of flat results. If we "do stuff," how do have any knowledge of whether it matters. Think of DIBELS. Roland had to match student overall reading performance with many demographic factors with student performance on PSF, NWF, ORF before he could state with some confidence that performance in these measures is highly correlated with reading and academic success.

We've invested heavily in some professional development and resources. We now need to look and see if there is some correlation. What could/would we monitor? Use of Open Court? Degree of effective implementation of Open Court? Evidence of thinking being taught/developed? Triangulation requires us to match it with student results- which do we value as a District in our primary kids. DRA? NWF? ORF? The evidence becomes more powerful if we gather it longitudinally- kids performance on NYS ELA in Grade three with and without OPen Court or its effective implementation.

In order to avoid the "rearview mirror" effect- we have to act. We have to do something different and implement it well. Antecedents absolutely have to do with teacher/adult behavior. It is up to us to determine what we feel is worth monitoring, get clear as a team about what, exactly we're monitoring and gather the data. Then we can match it by school or by teacher with learning using some of these tools to see if they matter for all kids or even certain types of kids. Without some triangulation, we'll stay in the "lucky/losing"side of the Reeves matrix because we'll have no real insight about whether what we're doing is working.

Ann makes a valid point- the use of antecedents represents a shift and is potentially an issue. We'll have to thoughtfully monitor, gather data and then use it with our team. Bringing it to teachers will require strong relationships to be in place and clear communication to help smooth the waters.SideBar

Tony:

I copied this from one of my comments from below:

How much time should be spent trying to do some of this very complex analysis. Shouldn't we try to determine a few (very few) effective work habits for teachers and then monitor those?

I understand that some of the tools will help us to evaluate results against certain work habits, but can't we start with a few simple, observable things and monitor them faithfully?

Authentic Literacy

What role does Schmoker's Concept of authentic literacy play in our improvement process?

Joyce Brace:

I am halfway through the Schmoker's section on authentic literacy and intellectual development. As usual, Schmoker gets right to the heart of the matter and presents a simple formula: Generous amounts of close, purposeful reading, rereading, writing and talking...

Ann and I were just talking about book clubs as part of the elementary reading block and we worry that they are often not purposeful. The examples in Appendix A are excellent and all of the Thoughtful Ed strategies and tools can be implemented to promote close, purposeful reading, rereading, writing and talking...

but how can we provide models for teachers. We need demonstration classrooms and coaches. So often teachers think that the book clubs they are running are providing opportunities for higher order thinking when they really aren't. They need to see it and then have someone who can observe and coach them through their own lessons.

The Authority to Act requires the Permission to Subtract

How can we understand and help teachers to understand what to subtract?

Joyce Brace:

I couldn't help thinking about Everyday Math when I read this in Beyond the Numbers. I have many teachers who believe that the program works with most kids but not with all kids. Do our teachers have the permission to "subtract" this math program from some students and use an alternative program? Also, check out the list of questions on page 92. Interesting...

The Power of Collaboration

What is the significance of the critical incident analysis?

Joyce Brace:

I really don't understand the stuff on page 61 of Beyond the Numbers? Can anyone help?

Jenifer Gilson:

My interpretation of this section, as well as much of the book, is that the author is providing a framework for much of what our inclinations might already be. That is, on pg. 61, he discusses critical incidents - the need to reflect on all aspects of schooling to identify priorities so that we can focus our efforts. This may be confusing because it is not what we traditionally think of as data analysis, but data is more than test scores/student results (quantitative data - numbers/scores). There is, also, qualitative data - observations, perceptions, descriptors of team work,etc. It is likely that we have looked at our buildings, teachers, and students and thought about 1) what creates the most difficulty for us/is most repetitious, 2) what specific projects cause the most problems/are most damaging to our reputation, 3) what are the most expensive things we do or require the greatest time and effort for the least return, and 4) what requires lots of duplicated effort/inhibits student achievement (Exhibit 4.1, p. 62). As principals, you probably consider these things all the time. White seems to be formalizing this contemplation so that administrators become conscious of their processes. That way we can support our teams to collaboratively contemplate the context in which they instruct and students learn. This identification of critical incidents set the stage for analyzing data that is less psychologically safe, i.e. more personal, and therefore, private. For example, your class performance on the state assessments, or on common formative assessments. This uncomfortable data, however, becomes necessary as teams verify they are on the right track with wanting to place the incidents they identified as critical. Lastly, this type of analysis helps to build the teams as they share ideas, validate each others' thinking, and develop a team dynamic. Does this help?

Work Habits and Change

What are your thoughts on this section (page 94 in Beyond the Numbers)?

Joyce Brace:

See my earlier comments on Triangulation and Intersections. How much time should be spent trying to do some of this very complex analysis. Shouldn't we try to determine a few (very few) effective work habits for teachers and then monitor those?

Replication and Action Research

Should we implement action research using a treatment group and a control group of students?