We caught up with a friend of ours this week. He told us he was working with a group of teachers who were committed to teaching journaling. Great! They were all excited about the project and they designed an inspiring plan. When he asked them how they would evaluate the project, they said they would use… the MAP assessment. Woah.
What is data REALLY and why does it matter so much?
Yes, data gives us much-needed information. Yes, it gives us great reports and data walls and all kinds of opportunities for analyzing performance. But data is actually every piece of observable information in your environment. It’s a huge pool of records, lessons, assessments, and meeting notes, it also includes students’ responses, conversations, facial expressions, what’s in the halls and on the walls. Your data pool is everything observable. It is not humanly possible to know and evaluate all of it, so we have to select what to focus on. However, that selection determines just about everything that happens from there.
- Data matters mostly because the data we select as our focus determines the language people use to describe the work.
- The language people use to describe the work determines how they practice.
- How people communicate and practice in their own environment determines the culture.
- Culture is the shared image that bonds the community of your school together and makes change possible.
The Ladder of Inference
How does this happen? There is a helpful graphic called The Ladder of Inference. Created by Chris Argyris and Peter Senge. It describes the thought process that we go through, usually without realizing it, to get from an observable fact to a decision or action.
Let’s look at how this ladder of thinking played itself out in our last cultural paradigm known as No Child Left Behind. We are not disparaging NCLB, its language, or its practices. Our intention is to demonstrate that the data we select leads us quickly up a ladder of thinking that significantly impacts school culture.
NCLB was implemented when we observed in the data pool that too many students were struggling to learn to read. So the National Reading Panel identified 5 Big Ideas.Performance on the 5 Big ideas became the data we selected.
We assumed that if we closed the gap on these 5 discrete skills, every student would learn to read. No Child would be Left Behind.
We concluded that investing in screeners, building intervention time into our schedules, hiring and training people to teach and assess, and implementing new programs would move us forward.
We developed a strong belief in this single measure system to the point where all other data points were necessarily obscured in comparison.
These beliefs led us to act on the belief that if “X” is good, then more “X” must be better. So, increasingly more time, people, and programs were added, all supporting a shared cultural image focused on this single measure system.
We believe NCLB was a step that we all learned a lot from, but it has reached its potential limit. It’s time to take the next step. The problem with NCLB is that, in too many cases, reading turned out to be a bit more complicated than we thought. The problem is not using single measures. The problem is using ONLY single measures.
The shift from data analysis to data evaluation
As a leader, you are tasked to transform your culture, but given little advice on concrete steps to take. Cultures evolve — they evolve through purposeful action. The first purposeful action you can take to change your culture for the better is to choose the data you want your people to FOCUS on.
You may not have the authority to decide WHAT data you use: it is usually (and most of the time rightfully) mandated. But you do have the freedom to select what data points you will focus on to inform and guide instruction. We have entered into a new stage of complexity, a stage that requires we use multiple, not single, measures.
We must give high stakes assessments. They tell us whether we are meeting expectations or not. We must give benchmarks. They tell us how we are doing compared to others. We must progress monitor. It tells us how much we need to improve. And now we must add one more piece: the assessment that tells us WHY a student is not meeting expectations. This assessment must be of authentic student work.
So what data should you select and why should it be your FOCUS?
Choose the data you VALUE.
- If you value journal writing…evaluate the JOURNALS.
- If you have invested in a new math program…evaluate the MATH WORK.
- If you value close reading…evaluate the TEXT ANNOTATION.
We have become so accustomed to solely rely on external sources that we have forgotten to appreciate the value of our intrinsic work. Again, these outside sources have their place. But when we depend on them without seriously considering the authentic assignments of our students, we can never answer the critical WHY. We forget that the very best predictor of student performance is the assignments we give them. Their authentic work makes their thinking visible. It informs our instruction better than any other measure. Best of all, “before” and “after” student work tells the most compelling stories about what is working and what is not.
We recommend spending 20% of your time evaluating external data sources and 80% of your time evaluating what you value most: the authentic work of your students. It’s the very best way to stop drowning in the data pool and start transforming your culture for the common good.
