In “Assessment of Reading Comprehension: Key Historical Influences in the USA,” Pearson and Hamm describe comprehension as a process, not an outcome:
“Comprehension, or ‘understanding’, by its very nature, is a phenomenon that can only be assessed, examined, or observed indirectly (Pearson and Johnson, 1978; Johnston, 1984a). We talk about the ‘click’ of comprehension that propels a reader through a text, yet we never see it directly. We can only rely on indirect symptoms and artifacts of its occurrence. People tell us that they understood, or were puzzled by, or enjoyed, or were upset by a text. Or, more commonly, we quiz them on ‘the text’ in some way – requiring them to recall its gist or its major details, asking specific questions about its content and purpose, or insisting on an interpretation and critique of its message. All of these tasks, however challenging or engaging they might be, are little more than the residue of the comprehension process itself.”
Given the claim that the comprehension process is only visible through its residue, Pearson and Hamm also reveal that we do not measure comprehension; we measure what people do with it.
This claim is further supported in Season 9, Episode 4 of Amplify’s The Science of Reading podcast. Hugh Catts claims that comprehension “is the interaction of what you bring into that reading situation and what you already know about it and your motivation and purpose to comprehend it.” For example, you might generate knowledge when reading a text that you then use to comprehend a subsequent text or you might develop an informed opinion about a topic or idea to then engage in a conversation with someone.
The Trouble with Standards
If reading is about use, then we stop short when we focus on mastering standards. Standards communicate finality, but they are really in the middle of a process. Often confused for the goal, I’m finally seeing this as a main issue with how we’ve designed reading comprehension instruction. Standards are guides along a process. They are not the end point.
Mastery of standards does not signal deep reading comprehension.
Take this released test item from a STAAR (Texas state assessment) Reading Language Arts (RLA) test.
Is “biohazard” at all related to overall comprehension of this text? Does it even matter or was it a convenient word to choose because it met a grade-level standard?
When I was designing assessments and then curriculum for the Louisiana Department of Education (LDOE), it was always a happy accident when a text had a feature that lined up well with some hard-to-meet standard. I recall many moments like this “biohazard” example where I would be able to check off a standard.
What is most irksome about a standards-based mindset is the erroneous assumption that if a comprehension lesson aligns to standards then it must mean students will understand the texts they’re reading. When the Common Core State Standards were released, educators pushed back on textbook publishers who created standards crosswalks and then stamped their existing textbooks with “Aligned to CCSS.” In 2013, we even made the bold claim at LDOE as part of our instructional materials reviews that no published textbooks were aligned to the CCSS. The claim at the time was that the standards required a different learning design and pedagogy, so simply aligning to the wording of standards wasn’t enough. This action opened the door for more open educational resources to be developed and adopted, which, in ELA, meant that more knowledge-building programs became available, such as the LDOE-produced ELA Guidebooks.
However, fast forward 15 years and many of our conversations have devolved from “Will this produce understanding and quality learning” to “Is this aligned to standards?” As I declare in “The System is Teaching the Test,” what gets measured is what matters and therefore taught. So, we teach standards.
What We Did Differently in Louisiana
When I created the Louisiana ELA Guidebooks, I eschewed standards. Our focus was for all students to read, understand, and express their understanding of complex texts. This served as a framework for a mindset shift: Instead of focusing on whether we can get students to master standards like “Analyze how a text makes connections among and distinctions between individuals, ideas, or events (e.g., through comparisons, analogies, or categories)” or “Determine an author’s point of view or purpose in a text and analyze how the author acknowledges and responds to conflicting evidence or viewpoints” (Common Core State Standards), we focused on selecting complex texts with these affordances. Our design was such that as teachers implemented the Guidebook units focused on understanding, they would need to help students engage in the knowledge and skills represented by standards because they were essential for comprehending a text.
We also focused on making sure that students made use of their comprehension to generate understanding. In a unit, students would read a series of texts and generate and refine their knowledge across them to then engage in a discussion about a broad concept, drawing on the knowledge they gained from the texts. Or, they would write an essay to answer a question about a concept or idea so as to contribute their voice to the conversation. In both of these examples, our focus was on “so what”? Why are students expressing their understanding? How does this develop their knowledge of words, the world, and/or themselves? How does this help build or refine their internal mental model to develop a deeper understanding of the world and their place in it?
In “What are We Really Teaching,” I made the case that understanding “is the point and purpose of all learning.” While I still agree with my past self, I’ve come to have a more nuanced understanding of understanding.
Our internal mental model is built from our knowledge and experiences with our environment, people, and concepts such as science, nature, technology, and history. We start building our theories about how the world works at an early age. All the developmental milestones that children have—rolling over, sitting up, babbling, eating solid foods, walking, talking, reading, writing, thinking logically to complete a puzzle, solving a math problem—are tied to their ability to observe, process, internalize, and then apply what they see in the world around them in their own unique way.
Comprehension is a critical step. It describes the cognitive process we engage in to make sense of what we read and observe. Comprehension is thinking. Our thoughts then lead the way to clarity and understanding, which is what we use to situate ourselves and move around in the world. Comprehension is an essential component of, rather than a synonym for, “understanding.”
And this process to develop comprehension is messy and often imprecise. Again, from “What are We Really Teaching”:
The words we use reveal our knowledge and understanding. If a child talks about conducting an experiment, chances are they have some concept of what experiments include, why they might conduct one, and how they might conduct one, even if there are still some gaps in their mental model.
Somewhere along the way we determined that a “gap” in a mental model was something to be fixed. If only learning were so predictable! Standards are meant to help us see how we can precisely predict and build learning for children. Standards are meant to be concrete, measurable, and obtainable. Standards for comprehension are an illusion. Meeting a standard does not mean comprehension or thinking occurred. This is the core difference between a standards-based and a comprehension-based implementation.
As my colleague at Learning Tapestry often says: “You can’t make the same decisions as Keith Richards, and then expect to be Keith Richards.” Likewise, we can’t teach the pieces and parts of reading and assume those discrete moves will somehow add up to comprehension. Comprehension and resulting understanding is bigger than standards alignment.
Why Standards Fail Us
What is that something more? In other words, why do the standards for reading comprehension fail us? In “When a Reader Meets a Text,” van den Broek, et. al. explain:
“A reader’s comprehension of a text depends on the processes that take place during reading. Some of these processes are automatic, others are strategic; some occur frequently, others occur only occasionally over the course of reading the text. Moreover, the processes evoked by the same text may differ from reader to reader or, for the same reader, across reading situations—for example, depending on the reader’s goals.”
van den Broek, et. al. further describe the processes which take place during reading, which comprise a reader’s comprehension:
“What determines the combination of processes that takes place as a reader proceeds through a text on a particular occasion? One important factor consists of text characteristics, such as the content and structure of the text at hand. Another important factor concerns the reader’s cognitive characteristics, such as his or her working-memory capacity, background knowledge, and so on. These factors have received considerable attention in research and are included in many models of text comprehension (McNamara & Magliano, 2009). In this chapter, we describe a third major factor that determines which processes are executed in an individual reading situation, namely the standards of coherence that the reader implicitly or explicitly adopts when reading the text.”
Based on this, there are three key factors which contribute to comprehension:
Text: The nature of complex text and the features which contribute to its complexity
Reader: The reader’s cognitive processing, including working and long-term memory, experiences and related knowledge, and schemas
Task: What the reader plans to do with their comprehension, which guides their expectations for how well they need to make sense of a text (“standards of coherence”) and what strategies they might deploy to support their expectations.
A reader’s standard of coherence for a text is based on what they expect to do with their comprehension. If a teacher asks students to read a book during “sustained silent reading” time but there’s no real purpose for the students to read, they may just flip through the pages, not actually reading. In this case, they would have a very low standard of coherence. There is really no expectation that the text will make sense and therefore, they do not engage cognitively with text features, they do not apply any of their knowledge, and they use no comprehension strategies to support their processing of the text. In this example, there is no comprehension, but then the students don’t expect any either, so their cognitive work (or lack thereof) aligns with their standards of coherence.
Conversely, if students are given a test with a reading passage, readers who are motivated to do well will have higher standards of coherence. Their higher standards of coherence will lead them to engage more cognitively in processing the text features, making connections to their experiences and prior knowledge, and using comprehension strategies as needed to help them engage cognitively with the text. Of course, if these students lack these abilities or knowledge, they will struggle to comprehend the text even as they might be motivated to comprehend it.
This leads to an important takeaway from the concept of “standards of coherence”: Comprehension is not a linear progression. We should not expect to see comprehension growth with every text we read. While the trend over time should be moving higher, the visual representation is more like a scatterplot graph than a straight line. Our level of comprehension will vary relative to the text, reader, and task or purpose for reading. And, depending on our expectations for something to make sense, we will deploy the appropriate strategies to ensure our expectations are met. Even then, comprehension may still break down.
Example 1: Einstein’s Relativity: The Special and General Theory
Take this excerpt from Einstein’s famous theory of relativity. (This is a highly complex excerpt.)
“It is not clear what is to be understood here by ‘position’ and ‘space.’ I stand at the window of a railway carriage which is travelling uniformly, and drop a stone on the embankment, without throwing it. Then, disregarding the influence of the air resistance, I see the stone descend in a straight line. A pedestrian who observes the misdeed from the footpath notices that the stone falls to earth in a parabolic curve. I now ask: Do the ‘positions’ traversed by the stone lie ‘in reality’ on a straight line or on a parabola? Moreover, what is meant here by motion ‘in space’? From the considerations of the previous section the answer is self-evident. In the first place we entirely shun the vague word ‘space,’ of which, we must honestly acknowledge, we cannot form the slightest conception, and we replace it by ‘motion relative to a practically rigid body of reference.’ The positions relative to the body of reference (railway carriage or embankment) have already been defined in detail in the preceding section. If instead of ‘body of reference’ we insert ‘system of co-ordinates,’ which is a useful idea for mathematical description, we are in a position to say: The stone traverses a straight line relative to a system of coordinates rigidly attached to the carriage, but relative to a system of co-ordinates rigidly attached to the ground (embankment) it describes a parabola. With the aid of this example it is clearly seen that there is no such thing as an independently existing trajectory (lit. ‘path-curve’ 1)), but only a trajectory relative to a particular body of reference.”
While you may have read that entire paragraph, how much did you comprehend? If I asked you, “Why does Einstein use the example of the railway carriage?” you might be able to share that he does so to explain his theory of relativity. But that does not mean that you comprehend his explanation. This excerpt has many complex features (e.g., sophisticated syntax, vague referents, high proposition density) and it requires a lot of prior knowledge related to mathematics and physics. Personally, I do not have well-established schemas to support many of the abstract concepts discussed. Very rarely does a breakdown of comprehension fall on a lack of strategy use, especially as strategies are about targeting areas to process the text features or connect to existing knowledge. Since I’m not able to cognitively engage with the text because of missing knowledge, being able to summarize a portion of the text doesn’t help me develop my comprehension. Visualizing the ideas might help, but my ability to develop a visual depends largely on my ability to comprehend the text, which is an obstacle. One possible way to help might be to ask an AI tool to summarize or produce a visual from this paragraph.
Example 2: Rosalyn Schanzer’s George vs. George
Now take this sentence from George vs. George, a text used in the Grade 4 ELA Guidebooks: “Not everyone realized it at the time, but the Revolutionary War had just begun.” (Access here: K-12 ELA Planning (under Language Tasks - Mentor Sentences))
Not only does this sentence include a main point of the overall reading passage, but generating meaning from this sentence engages students in parsing complexity in syntax and ideas. Focusing on comprehension of sentences, both understanding their meaning AND their importance in the overall text, models for them how to approach sentences in complex text to derive meaning.
What makes this sentence difficult to comprehend? What features might a teacher help a student understand to make meaning when reading this sentence?
The sentence contains two clauses with different functions (Dependent clause + conjunction + independent clause).
The main part of the sentence is at the end; the first part sets up a contrast, which adds complexity to the ideas expressed.
Readers have to hold two contrasting ideas simultaneously: What people knew or noticed at the time and what was actually happening.
Readers must infer that the events at the time were not war-like and the author of this sentence is aware of that information.
“It” comes before the antecedent “Revolutionary War,” requiring readers to resolve meaning backwards.
To teach this sentence with a focus on comprehension rather than mastery of standards, first determine whether students can parse this sentence. Ask questions such as:
“Why might people have been unaware of the Revolutionary War beginning?”
“How do we view events now compared to how they felt at the time? What does this reveal about how knowledge changes over time?”
“Why is the revelation of this sentence important for understanding how we think about and study history?”
If students demonstrate a lack of comprehension, follow up with more direct modeling, such as:
“What does ‘it’ mean in this sentence?”
“What does ‘but’ signal between the two parts of the sentence?”
“Did more or fewer people understand the war was starting?”
Teach for Comprehension, Not Standards
Why is moving toward comprehension-based instruction and away from standards-based instruction important? As I shared in “What are We Really Teaching”:
“In education, we often lose sight of why we’re engaged in teaching and learning. We substitute the values and expectations of leaders, participation in a system, learning standards, and assessments as the reason for learning:
“My district told me that I have to teach this.”
“A law was passed that requires me to teach this way.”
“I want my kids to be prepared for the next grade.”
“I need to know that they can do what is detailed in this standard.”
“I want them to pass the test so they can move on to the next grade and so that I can keep my job.”
But all of these reasons simply dilute the real reason we’re teaching: to help students create their own internal mental models of the world around them.
If we want students to truly develop understanding, they must be able to comprehend complex texts. Instead of asking: “What standards do these questions address?” ask: “What do students need to understand here, and how do I support them in the thinking needed?” If you focus on mastering standards and assume that students will magically develop deep comprehension, you will miss the mark.
Takeaways
Comprehension is a cognitive process, not an outcome.
We do not measure comprehension; we measure what we do with our comprehension.
Mastery of standards does not signal deep reading comprehension.
Comprehension is an essential component of, rather than a synonym for, “understanding.”
Comprehension and resulting understanding is bigger than standards alignment.
Comprehension is not a linear progression.
References:
Pearson, P. David & Hamm, Diane Nicole. (2006). The assessment of reading comprehension: Key historical influences in the USA. In Assessing Reading: From theories to classrooms (pp. 76-101). National Foundation for Educational Research.
Lambert, S. (Host). (Season 9, Episode 4). Comprehension is not a skill, with Hugh Catts, Ph.D. [Audio podcast episode]. In Science of Reading: The Podcast. Amplify. https://amplify.com/episode/science-of-reading-the-podcast/season-9/episode-4-comprehension-is-not-a-skill-with-hugh-catts-ph-d/
van den Broek , Paul & Bohn-Gettler, Catherine & Kendeou, P. & Carlson, Sarah & White, M.J.. (2011). When a reader meets a text: The role of standards of coherence in reading comprehension. Text relevance and learning from text. 123-140.
Einstein, A. (1920). Relativity: The special and the general theory (R. W. Lawson, Trans.). Methuen & Co. Ltd..
Schanzer, R. (2004). George vs. George: The American Revolution as seen by both sides. National Geographic Children’s Books.
Well said. You're right, too many teachers/administrators look at a list of standards and think, "Here is what we have to cover," rather than, "What is most valuable for my students to know, and how do I get them there?"
The same is true in Math education. In the lower grades, students are supposed to "understand" when this is not measurable. Meanwhile, their numeracy skills are never developed and the deeper understanding that comes from thinking applied to learned memories is truncated.