«Text-based recall and extra-textual generations resulting from simplified and authentic texts Scott A. Crossley Georgia State University United ...»
Reading in a Foreign Language April 2016, Volume 28, No. 1
ISSN 1539-0578 pp. 01–19
Text-based recall and extra-textual generations resulting from simplified and
Scott A. Crossley
Georgia State University
Danielle S. McNamara
Arizona State University
This study uses a moving windows self-paced reading task to assess text comprehension of beginning and intermediate-level simplified texts and authentic texts by L2 learners engaged in a text-retelling task. Linear mixed effects (LME) models revealed statistically significant main effects for reading proficiency and text level on the number of text-based propositions recalled: More proficient readers recalled more propositions. However, text level was a stronger predictor of propositional recall than reading proficiency. LME models also revealed main effects for language proficiency and text level on the number of extra-textual propositions produced. Text level, however, emerged as a stronger predictor than language proficiency. Post-hoc analyses indicated that there were more irrelevant elaborations for authentic texts and intermediate and authentic texts led to a greater number of relevant elaborations compared to beginning texts.
Keywords: text readability, text comprehension, L2 reading, text simplification Second language (L2) readers and teachers generally have two choices when selecting reading texts: authentic texts that were developed for first language (L1) readers or texts that have been linguistically simplified to increase comprehension. There are obvious trade-offs between the two choices and neither is optimal. For instance, authentic texts, while preserving natural language complexity and cultural relevance, are often difficult to process and comprehend because of their use of lexically sophisticated words and chunks, syntactic complexity, and lack of explicit cohesive devices (Crossley, Allen, & McNamara, 2011, 2012; Crossley, Louwerse, McCarthy, & McNamara, 2007; Crossley & McNamara, 2008). Simplified texts, on the other hand, appear to be easier to process and comprehend because of the manipulation of linguistic features (Crossley, Yang, & McNamara, 2014; Long & Ross, 1993; Oh, 2001; Tweissi, 1998;
Yano, Long, & Ross, 1994), but the process of simplification can rob the texts of their natural rhythm and cultural significance (Little, Devitt, & Singleton, 1989; Long & Ross, 1993).
http://nflrc.hawaii.edu/rfl Crossley & McNamara: Text-based recall and extra-textual generations 2 One limitation on the previous behavioral studies conducted on text simplification and its relation to text comprehension has been the manner in which comprehension has been defined.
The majority of previous studies have measured comprehension through comprehension questions (e.g., true/false or multiple choice questions; Crossley et al., 2014; Long & Ross, 1993;
Oh, 2001; Tweissi, 1998; Yano, Long, & Ross, 1994). While comprehension questions provide an indication of comprehension, they have limitations. These limitations include the notions that comprehension questions generally query only a small number of the ideas found in a text, can be correctly guessed (Day & Park, 2005), and do not reflect theoretical assumptions guided by comprehension models (Kintsch, 1988, 1998).
In this study, we examine text comprehension at various text levels (authentic texts and texts simplified to the beginning and intermediate levels) using a text-retelling paradigm that is embedded in a self-paced reading experiment. Text retelling allows readers to freely produce the propositions they recall from reading the text as well as extra-textual elaborations. The number of propositions recalled is limited only by the time available for the retelling, and retellings by their very nature do not allow readers to guess at their answer. In addition, the use of propositions as a measure of comprehension is firmly rooted in a number of theoretical and empirical accounts of reading (e.g., the Construction-Integration model of comprehension;
Kintsch, 1988, 1998). Thus, our goal in this study is to assess the relations between text simplification and comprehension in L2 readers using propositional data. In addition, we examine additional non-textual factors that are often strongly related to reading comprehension including reader background knowledge, reading proficiency, and overall L2 language proficiency (Crossley et al., 2014). These factors have been generally neglected in previous research on the effects of text simplification (e.g., Long & Ross, 1993; Oh, 2001; Tweissi, 1998;
Yano et al., 1994). Such an approach allows us to answer the following research questions:
1. Are there differences in text comprehension as measured by propositions recalled for L2 readers among texts simplified to the beginning and intermediate level and authentic texts?
2. Does an L2 reader’s background knowledge, language proficiency, or reading proficiency aid in text comprehension?
3. Do texts simplified to the beginning and intermediate levels lead to a greater or smaller number of extra-textual propositions produced as compared to authentic texts?
4. Does an L2 reader’s background knowledge, language proficiency, or reading proficiency lead to a greater or smaller number of extra-textual propositions?
Authentic texts are unmodified texts that were originally created to fulfill a social purpose in a first language community (Little, Devitt, & Singleton, 1989). Often authentic texts are modified to make them more linguistically accessible for L2 readers. In this way, material developers hope to maintain the cultural relevance of the text while, at the same time, simplifying the text to make it more comprehensible. Such text modifications generally occur at the syntactic and lexical level (Hill, 1997), but modifications are also common at the level of cohesion (Crossley, Louwerse, McCarthy, & McNamara, 2007; Crossley & McNamara, 2008). Some authentic texts are also Reading in a Foreign Language 28(1) Crossley & McNamara: Text-based recall and extra-textual generations 3 simplified through elaboration in order to clarify the content of the text and simplify the text structure through the repetition of key ideas and the paraphrasing of difficult terms (Yano, Long, & Ross, 1994), although such elaboration appears to lead to decreased readability (Long & Ross, 1993; McNamara, Kintsch, Songer, & Kintsch, 1996).
While there are many approaches to text simplification, such as adapting or abridging original texts and writing texts specifically to practice a grammar or linguistic form, all simplified texts share the same goal: reducing the cognitive load and increasing text comprehensibility on the part of the L2 reader. When simplifying a text, material developers generally follow two approaches: a structural approach or an intuitive approach (Allen, 2009). In an intuitive approach, authors use their experiences as a language teacher, language learner, and/or materials developer to guide them in the process of text simplification. Thus, an intuitive approach relies on an author’s subjective judgment of what learners at a particular level are able to comprehend and read (Allen, 2009). A structural approach to simplification relies on authors using pre-defined word and structure lists. These approaches are most commonly used in graded reader texts that are linked to practices of extensive reading. In a similar fashion, authors may rely on traditional readability formulas that assess text readability based on sentence length and word length to simplify text. While such readability formulas can be successful at predicting L1 text readability, they are widely criticized as weak indicators of comprehensibility (Carrell, 1987; Crossley, Greenfield, & McNamara, 2008; Davison & Kantor, 1982). Of these two approaches to text simplification (intuitive and structural), intuitive approaches are more common (Crossley, Allen, & McNamara, 2012; Simensen, 1987).
Simplification and Textual Effects
The reasons behind text simplification are clearly defined. However, the linguistic effects of such modifications on texts were unclear until recently. That is to say, material developers routinely simplified texts in order to make them more readable and comprehensible, but to what degree these modifications led to linguistic differences as compared to authentic texts remained uncertain. In a series of studies conducted by Crossley and colleagues, the linguistic differences between authentic and simplified texts (Crossley et al., 2007; Crossley & McNamara, 2008) and between levels of simplified text (Crossley, Allen, & McNamara, 2011, 2012) were clarified.
These studies generally supported the notion that the process of text simplification led to significant changes in the linguistic structure of texts, both when comparing simplified to authentic texts and when comparing levels of simplified texts. The findings provided evidence that simplification should lead to texts being easier to read and comprehend. For example, Crossley et al. (2007) and Crossley and McNamara (2008) reported that authentic texts used for beginning and intermediate L2 learners were syntactically more complex, contained a greater density of logical connectors, contained greater lexical sophistication (e.g., more infrequent words, less specific words, words with more senses, and less familiar words) and had lower levels of cohesion (e.g., less lexical co-reference and semantic overlap) than simplified texts used at the same levels. In reference to texts simplified to specific levels (i.e., text simplified for advanced, intermediate, and beginning level L2 readers), Crossley et al. (2012) found that advanced level simplified texts when compared to beginning simplified texts were more complex Reading in a Foreign Language 28(1) Crossley & McNamara: Text-based recall and extra-textual generations 4 lexically (e.g., contain greater lexical diversity, more infrequent words, more unfamiliar words, and less concrete words), syntactically (e.g., have less syntactic similarity and more words before the main verb), and cohesively (e.g., less given information, less semantic co-referentiality, and less noun overlap). These studies indicate that the process of simplification leads to the creation of texts that should be easier to process and comprehend for L2 readers.
Simplification and Text Comprehension
While linguistic differences between simplified and authentic texts and differences between texts simplified to various proficiency levels are indicative of potential processing differences, they do not provide evidence of processing differences. For that, behavioral studies are needed. Those behavioral studies that have examined the effects of text simplification on L2 readers have generally supported the notion that simplified texts do lead to both faster reading times and improved text comprehension. For instance, Yano et al. (1994) reported that simplified texts as compared to authentic texts, increased text comprehension. In more recent studies, Tweissi (1998) and (Oh, 2001) also reported that simplification positively affected L2 students’ overall reading comprehension. However, at least one study (Long & Ross, 1993) indicates complications with text simplification that raise cautionary notes. Similar to other studies, Long and Ross (1993) reported that texts linguistically simplified using traditional readability formulas led to greater comprehension in L2 readers when compared to authentic texts. However, Long and Ross also reported that readers’ English proficiency level and reading comprehension scores affected text comprehension with higher proficiency learners and readers exhibiting better text comprehension.
While these studies collectively support the use of simplified over authentic texts in terms of text comprehension, potential limitations in their experimental designs indicate that the results should be interpreted with caution. For instance, the Tweissi (1998) study did not statistically control for potential linguistic differences between text conditions and Long and Ross (1993) and Yano et al.
(1994) relied solely on traditional readability formulas, which are limited in the number of linguistic features they measure, to assess differences between simplified and authentic text.
More importantly, many of the studies did not control for reading proficiency (Long & Ross, 1993; Oh, 2001; Tweissi, 1998; Yano et al., 1994), language proficiency (Yano et al., 1994;
Tweissi, 1998), or background knowledge (Long & Ross, 1993; Oh, 2001; Tweissi, 1998; Yano et al., 1994) when assessing text comprehension. Reading and language proficiency (Buswell,
1922) along with background knowledge are important predictors of readability and text comprehension (McNamara et al., 1996; Shapiro, 2004).
To at least partially address these limitations, Crossley et al. (2014) used a moving windows selfpaced reading task to examine differences in reading times and comprehension for L2 learners reading authentic texts and texts simplified to the beginning and intermediate levels. In addition to controlling for linguistic differences in the text using the computational tool Coh-Metrix (Graesser, McNamara, Louwerse, & Cai, 2004; McNamara & Graesser, 2012; McNamara, Graesser, McCarthy, & Cai, 2014), Crossley et al. also controlled for the reading proficiency, language proficiency, and background knowledge of the L2 participants. Crossley et al. used a moving windows self-paced reading task in order to simulate eye movement data (Just, Carpenter, & Woolley, 1982).