Data Collection for Peer Visitation
Data and Data Collection
Our own experiences as educators make it challenging to shed our perceptions about instructional methods (or even particular students!) when we enter the classroom as observers. Although prior experiences are valuable, they can inhibit a teacher in the observer role. He or she may draw conclusions before taking the time to make sense of all of the data collected.
Sometimes, teachers make comments to other teachers like, "Wow, your students were really engaged!" or "That work was too difficult for those students." Such general comments don't easily inform shifts in instructional practice. They more often result in a failure to identify instructional improvements or in defensiveness. In addition, educators are in the habit of drawing immediate conclusions about our classrooms—we do so hundreds of times a day as we make decisions.
Quality teacher-driven observation requires that we take a moment to pause and examine the data collected in our classrooms. Our own experiences as educators make it challenging to shed our perceptions about instructional methods (or even particular students) when we enter the classroom as observers. Although prior experiences are valuable, they can inhibit a teacher in the observer role. He or she may draw conclusions before taking the time to make sense of all of the data collected.
Staying descriptive is the single most difficult skill to develop in teacher-driven observation. Too often adequate training and support is not given to teachers on how to do this skill well. Because you are a teacher does not mean you don't need any practice or guidance giving this type of observational feedback. It's quite powerful when a visiting teacher stays descriptive throughout the post-observation debriefing. Rather than drawing a conclusion about student engagement, for instance, the observing teacher discusses exactly what he or she saw and heard in the classroom. The pool of data he or she shares through descriptive evidence lays a richer foundation for discussion.
Sometimes, teachers make comments to other teachers like, "Wow, your students were really engaged!" or "That work was too difficult for those students." Such general comments don't easily inform shifts in instructional practice. They more often result in a failure to identify instructional improvements or in defensiveness. In addition, educators are in the habit of drawing immediate conclusions about our classrooms—we do so hundreds of times a day as we make decisions.
Quality teacher-driven observation requires that we take a moment to pause and examine the data collected in our classrooms. Our own experiences as educators make it challenging to shed our perceptions about instructional methods (or even particular students) when we enter the classroom as observers. Although prior experiences are valuable, they can inhibit a teacher in the observer role. He or she may draw conclusions before taking the time to make sense of all of the data collected.
Staying descriptive is the single most difficult skill to develop in teacher-driven observation. Too often adequate training and support is not given to teachers on how to do this skill well. Because you are a teacher does not mean you don't need any practice or guidance giving this type of observational feedback. It's quite powerful when a visiting teacher stays descriptive throughout the post-observation debriefing. Rather than drawing a conclusion about student engagement, for instance, the observing teacher discusses exactly what he or she saw and heard in the classroom. The pool of data he or she shares through descriptive evidence lays a richer foundation for discussion.
Descriptive, Low-Inference Evidence (without observer's judgment) and Non-Descriptive, High-Inference Evidence (includes observer's judgment)
Examples of Descriptive Evidence (without observer's judgment)
- Teacher asks, "How did you figure out this problem?" Student explains.
- Task: Find different ways to create a total of 31.
5+5+5+5+5+5+1
10+10+10+1
Student 2 wrote in math journal:
20+9=03
Student 3 wrote in math journal:
41-10=31
2+3x3+16=31
- Matt asks Sue: "What are we supposed to write down?"
Examples of Non-Descriptive Evidence (includes observer's judgment)
- Fast-paced.
- Too much time on discussion, not enough time on individual work.
- Excellent classroom management.
- Teacher used effective questioning technique with a range of students.
- Teacher had good rapport with students.
- Students conducted a very sophisticated lab experiment.
- Teacher read from a book that was not at the appropriate level for the class.
Why Use Low-Inference Notes and Evidence?
Synthesizing more than 900 educational meta-analyses, researcher John Hattie has found that effective feedback is among the most powerful influences on how people learn.
(John Hattie, Know Thy Impact, Educational Leadership Feedback for Learning, September 2012, Vol. 70, No. 1)
Using low-inference feedback…
“The term feedback is often used to describe all kinds of comments made after the fact, including advice, praise, and evaluation. But strictly speaking, none of these are feedback.
Basically, feedback is information about how we are doing in our efforts to reach a goal. Even further, helpful feedback is goal-referenced; tangible and transparent; actionable; user-friendly (specific and personalized); timely; ongoing; and consistent.”
- Grant Wiggins, Feedback for Learning: September, 2012; Vol. 70, No.1
In order to become efficient and effective at providing low inference feedback both during conferencing and in written commentary, a teacher must first become proficient at completing and recording low inference notes during the observation period.
(John Hattie, Know Thy Impact, Educational Leadership Feedback for Learning, September 2012, Vol. 70, No. 1)
Using low-inference feedback…
“The term feedback is often used to describe all kinds of comments made after the fact, including advice, praise, and evaluation. But strictly speaking, none of these are feedback.
Basically, feedback is information about how we are doing in our efforts to reach a goal. Even further, helpful feedback is goal-referenced; tangible and transparent; actionable; user-friendly (specific and personalized); timely; ongoing; and consistent.”
- Grant Wiggins, Feedback for Learning: September, 2012; Vol. 70, No.1
In order to become efficient and effective at providing low inference feedback both during conferencing and in written commentary, a teacher must first become proficient at completing and recording low inference notes during the observation period.
Three Distinct Methods of Data Collection
Because classroom data collection is at the heart of teacher-driven observation, it's crucial that observers are clear about the type of data they should collect. Here are three distinct methods of data collection:
Scripting. This method requires observers to transcribe interactions between the teacher and students—things like teacher instructions and questions, student answers, or individual conversations—as well as interactions among students. Student answers may be further categorized as voluntary, in response to being called on, related to the lesson, or supported (or unsupported) by pertinent details. Scripting engenders a wealth of detail and so is relevant to nearly every observation focus area.
However, recording such a quantity of data can be challenging for an observer. The teacher may elect to narrow the focus, asking observers to script the questions he asks, the responses students give, the interaction among a small group of kids, or the dialogue between the teacher and certain students.
However, recording such a quantity of data can be challenging for an observer. The teacher may elect to narrow the focus, asking observers to script the questions he asks, the responses students give, the interaction among a small group of kids, or the dialogue between the teacher and certain students.
Counting. Counting methods let observers capture a high-level view of one particular element of the classroom. A teacher may use counting to examine the distribution of class time, the number of questions she asks at each level of Bloom's taxonomy, or the percentage of students who voluntarily respond to questions.
Although counting methods provide insight into classroom patterns, they're often best paired with scripting or tracking methods to provide deeper insight. For example, a teacher may want to explore the number of comments from each student during a whole-class discussion alongside the scripted comments themselves.
Although counting methods provide insight into classroom patterns, they're often best paired with scripting or tracking methods to provide deeper insight. For example, a teacher may want to explore the number of comments from each student during a whole-class discussion alongside the scripted comments themselves.
Tracking. Tracking can also help illustrate patterns in a classroom. An observer may use tracking methods to collect data on movement in a classroom—such as by recording on a map of the classroom the movement of a teacher during a lesson or of students during transition time. An observer might track students' eye contact to examine patterns in classroom attention.
Classrooms are complex. Observed teachers often select multiple data-collection methods to gain a holistic picture of their focus area. As the leader of teacher-driven observation, the observed teacher selects the data-collection methods an observer will use. The method or methods should align with that teacher's area of inquiry and focus question.
Low-Inference Notes
There is a significant difference between low-inference evidence and opinion/interpretation. Low-inference note-taking helps observers accurately interpret teacher practice in order to support teacher development.
What are low-inference notes?
Low-inference notes describe what is happening without drawing conclusions or making making judgments.
Time Teacher Actions Student Actions
1:01 Teacher asked students to stand 16 out of 27 stood up for
and led them in the "Days of the Week" song
song
1:03 Teacher asked, "What day comes after Steven shouted out,
Saturday? "Monday." Most of the
students laughed. Two
boys laughing rolled around on the floor
knocking over two girls.
Steven walked away from
the group and sat on the
opposite side of the
classroom.
Example of visitation notes that are not low-inference notes
Time Teacher Actions Student Actions
1:01 Teacher reviewed the days of the Many students were not
week. listening.
1:03 Teacher asked the question about Steven called out over and
the days of the week. over again.
What are low-inference notes?
Low-inference notes describe what is happening without drawing conclusions or making making judgments.
- What do you see and hear the teacher and students doing?
- What evidence can you gather of student learning?
- What will students know and be able to do at the end of the lesson?
Time Teacher Actions Student Actions
1:01 Teacher asked students to stand 16 out of 27 stood up for
and led them in the "Days of the Week" song
song
1:03 Teacher asked, "What day comes after Steven shouted out,
Saturday? "Monday." Most of the
students laughed. Two
boys laughing rolled around on the floor
knocking over two girls.
Steven walked away from
the group and sat on the
opposite side of the
classroom.
Example of visitation notes that are not low-inference notes
Time Teacher Actions Student Actions
1:01 Teacher reviewed the days of the Many students were not
week. listening.
1:03 Teacher asked the question about Steven called out over and
the days of the week. over again.
Are These Examples of Low-Inference, Quality Notes?
Which are written in low inference and which are written in high inference?
How can you tell?
How can we change the high inference statement into low-inference statements?
1. The teacher asked 3 clarifying questions to the pair of students.
2. One child did not understand the assignment.
3. There are five students who had their heads down.
4. This student is an English Language Learner.
5. Two causes for the American Revolution were listed and discussed.
How can you tell?
How can we change the high inference statement into low-inference statements?
1. The teacher asked 3 clarifying questions to the pair of students.
2. One child did not understand the assignment.
3. There are five students who had their heads down.
4. This student is an English Language Learner.
5. Two causes for the American Revolution were listed and discussed.
Answers:
1. The teacher asked 3 clarifying questions to the pair of students.
This could be considered a low inference statement because it is a factual observation, but it is not high quality. However, one could record information about how the students answered the questions…this will help understand the learning that is going on in the classroom.
2. One child did not understand the assignment.
--High inference: Make adjustments such as noting what the teacher and student did. This statement is interpretation of what is taking place. Record the student questions or record teacher – student interactions for clarity.
3. There are five students who had their heads down.
–This is a low inference, factual statement.
4. This student is an English Language Learner.
–This is a low inference, factual statement.
5. Two causes for the American Revolution were listed and discussed.
–Although this is a factual recording, it is not high quality. If more information were included about the student interaction and a transcript or excerpt of the discussion, it would allow for more specific feedback.
Reflection:
Why is it important to collect low-inference evidence before trying to assess teacher practice?
How can low-inference evidence support teacher development?•
This could be considered a low inference statement because it is a factual observation, but it is not high quality. However, one could record information about how the students answered the questions…this will help understand the learning that is going on in the classroom.
2. One child did not understand the assignment.
--High inference: Make adjustments such as noting what the teacher and student did. This statement is interpretation of what is taking place. Record the student questions or record teacher – student interactions for clarity.
3. There are five students who had their heads down.
–This is a low inference, factual statement.
4. This student is an English Language Learner.
–This is a low inference, factual statement.
5. Two causes for the American Revolution were listed and discussed.
–Although this is a factual recording, it is not high quality. If more information were included about the student interaction and a transcript or excerpt of the discussion, it would allow for more specific feedback.
Reflection:
Why is it important to collect low-inference evidence before trying to assess teacher practice?
How can low-inference evidence support teacher development?•
How Can Low-Inference Data Collection and Skills be Learned?
• Providing low inference feedback will require practitioners to acquire a new skill.
• As with any skill it gets easier with practice.
• Typically, novice transcribers struggle with: speed, accuracy and stamina.
• As with any skill it gets easier with practice.
• Typically, novice transcribers struggle with: speed, accuracy and stamina.