Adaptive Literacy Instruction in iSTART and W-Pal: Implementing the Inner and Outer Loop
Amy M. Johnson, Cecile A. Perret, Micah Watanabe, Kristopher J. Kopp, Kathryn S. McCarthy, and Danielle S. McNamara
Corresponding Author: firstname.lastname@example.org
Though the affordances of intelligent tutoring systems (ITSs) are particularly useful for literacy instruction, developing accurate automated feedback in reading and writing present challenges for developers. This chapter outlines some of these challenges in the context of providing both inner-loop and outer-loop feedback (VanLehn, 2006) and describes how we have addressed these challenges in our systems, iSTART and W-Pal. We report on the theoretical and practical development of student and instructional models that are sensitive to students’ evolving literacy skills. The chapter offers an explanation of how we posit the implementation of outer-loop adaptivity can enhance learning of literacy skills in ITSs more broadly.
Keywords: inner and outer loop adaptivity, ill-defined domains, natural language processing, literacy instruction, pedagogical design
APA citation information
Johnson, A. M., Perret, C. A., Watanabe, M., Kopp, K. J., McCarthy, K. S., & McNamara, D. S. (2018). Adaptive literacy instruction in iSTART and W-Pal: Implementing the inner and outer loop. In S. D. Craig (Ed.). Tutoring and Intelligent Tutoring Systems (pp. 221-249). New York, NY: Nova Science Publishers.
Alexander, P. A., Graham, S., & Harris, K. R. (1998). A perspective on strategy research: Progress and prospects. Educational Psychology Review, 10(2), 129-154.
Allen, L. K., Crossley, S. A., Snow, E. L., & McNamara, D. S. (2014). Game-based writing strategy tutoring for second language learners: Game enjoyment as a key to engagement. Language Learning and Technology, 18, 124-150.
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. The journal of the learning sciences, 4(2), 167-207.
Balyan, R., McCarthy, K. M., & McNamara, D. S. (in press). Comparing Machine Learning Classification Approaches for Predicting Expository Text Difficulty. In Proceedings of the 31st Annual Florida Artificial Intelligence Research Society International Conference (FLAIRS).
Beck, I. L., McKeown, M. G., & Gromoll, E. W. (1989). Learning from social studies texts. Cognition and instruction, 6(2), 99-158.
Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe and A. P. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 185-205). Cambridge, MA: MIT Press.
Bransford, J. D., & Johnson, M. K. (1972). Contextual prerequisites for understanding: Some investigations of comprehension and recall. Journal of verbal learning and verbal behavior, 11(6), 717-726.
Chi, M. T., & Bassok, M. (1989). Learning from examples via self-explanations. In L. B. Resnick (Ed.), Knowing, learning, and instruction: Essays in honor of Robert Glaser (pp. 251-282). Hillsdale, NJ: Lawrence Erlbaum Associates
Chi, M. T., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self‐explanations: How students study and use examples in learning to solve problems. Cognitive science, 13(2), 145-182.
Chi, M. T., de Leeuw, N., Chiu, M. H., & LaVancher, C. (1994). Eliciting self‐explanations improves understanding. Cognitive science, 18(3), 439-477.
Collins, A., Brown, J. S., & Newman, S. E. (1988). Cognitive apprenticeship: Teaching the craft of reading, writing and mathematics. Thinking: The Journal of Philosophy for Children, 8(1), 2-10.
Condon, W. (2013). Large-scale assessment, locally-developed measures, and automated scoring of essays: Fishing for red herrings?, 18, 100–108.
Council of Writing Program Administrators, National Council of Teachers of English, & The National Writing Project. (2011). Framework for success in postsecondary writing. Retrieved from http://wpacouncil.org/files/framework-for-success-postsecondary-writing.pdf.
Crawford, L., Lloyd, S., & Knoth, K. (2008). Analysis of student revisions on a state writing test. Assessment for Effective Intervention, 33(2), 108-119.
Crossley, S. A., Allen, D., & McNamara, D. S. (2011). Text readability and intuitive simplification: A comparison of readability formulas. Reading in a Foreign Language, 23, 84-102.
Crossley, S. A., Allen, L. K., & McNamara, D. S. (2016). The Writing Pal: A writing strategy tutor. In S. A. Crossley & D. S. McNamara (Eds.) Adaptive educational technologies for literacy instruction (pp. 204-224). New York: Taylor & Francis, Routledge.
Crossley, S. A., Dempsey, K., & McNamara, D. S. (2011). Classifying paragraph types using linguistic features: Is paragraph positioning important? Journal of Writing Research, 3, 119-143.
Crossley, S. A., Roscoe, R. D., & McNamara, D. S. (2014). What is successful writing? An investigation into the multiple ways writers can write high quality essays. Written Communication, 31, 181-214.
Elbow, P. (1973). Writing Without Teachers. New York: Oxford Press.
Elliot, N., & Klobucar, A. (2013). Automated essay evaluation and the teaching of writing. In M. D. Shermis, J. Burstein, & S. Apel (Eds.), Handbook of automated essay evaluation: Current applications and new directions (pp. 16–35). London: Routledge.
Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological review, 100(3), 363.
Fitzgerald, J. (1987). Research on revision in writing. Review of Educational Research, 57(4), 481-506.
Flower, L., & Hayes, J. (1981). Identifying the organization of writing processes. In L. Gregg and E. Steinberg (Eds.), Cognitive Processes in Writing (pp. 3-30). Hillsdale, NJ: Erlbaum & Associates.
Flowerday, T., & Schraw, G. (2000). Teacher beliefs about instructional choice: A phenomenological study. Journal of educational psychology, 92(4), 634.
Graesser, A. C., Chipman, P., Haynes, B. C., & Olney, A. (2005). AutoTutor: An intelligent tutoring system with mixed-initiative dialogue. IEEE Transactions on Education, 48(4), 612-618.
Graham, S., & Perin, D. (2007). A meta-analysis of writing instruction for adolescent students. Journal of Educational Psychology, 99(3), 445–476.
Healy, A. F., King, C. L., Clawson, D. M., Sinclair, G. P., Rickard, T. C., Crutcher, R. J., … Bourne, L. E., Jr. (1995). Optimizing the long-term retention of skills. In A. F. Healy & L. E. Bourne, Jr. (Eds.), Learning and memory of knowledge and skills: Durability and specificity (pp. 1-29). Thousand Oaks, CA, US: Sage Publications, Inc.
Jackson, G. T., Boonthum, C., & McNamara, D. S. (2009). iSTART-ME: Situating extended learning within a game-based environment. In H. C. Lane, A. Ogan, & V. Shute (Eds.), Proceedings of the Workshop on Intelligent Educational Games at the 14th Annual Conference on Artificial Intelligence in Education (pp. 59-68). Brighton, UK: AIED.
Jackson, G. T., & McNamara, D. S. (2013). Motivation and performance in a game-based intelligent tutoring system. Journal of Educational Psychology, 105, 1036-1049.
Jackson, G. T. & McNamara, D. S. (2017). The motivation and mastery cycle framework: Predicting long-term benefits of educational games. In Y. Baek (Ed.), Game-based learning: Theory, strategies and performance outcomes. (pp. 97 – 122). Nova Science Publishers: New York.
Kellogg, R. T. (2008). Training writing skills: A cognitive developmental perspective. Journal of writing research, 1(1), 1-26.
Kintsch, W. (1988). The role of knowledge in discourse comprehension: A construction-integration model. Psychological review, 95(2), 163.
Kintsch, W. (1998). Comprehension: A paradigm for cognition. Cambridge university press.
McNamara, D. S. (2004). SERT: Self-explanation reading training. Discourse Processes, 38, 1-30.
McNamara, D. S., Boonthum, C., Levinstein, I. B., & Millis, K. (2007). Evaluating self-explanations in iSTART: Comparing word-based and LSA algorithms. In T. Landauer, D. S. McNamara, S. Dennis, & W. Kintsch (Eds.), Handbook of Latent Semantic Analysis (pp. 227-241). Mahwah, NJ: Erlbaum.
McNamara, D. S., Crossley, S. A., & Roscoe, R. D. (2013). Natural language processing in an intelligent writing strategy tutoring system. Behavior Research Methods, 45, 499-515.
McNamara, D. S., Crossley, S. A., Roscoe, R. D., Allen, L. K., & Dai, J. (2015). Hierarchical classification approach to automated essay scoring. Assessing Writing, 23, 35-59.
McNamara, D. S., & Graesser, A. C. (2012). Coh-Metrix: An automated tool for theoretical and applied natural language processing. In P. M. McCarthy & C. Boonthum-Denecke (Eds.), Applied natural language processing and content analysis: Identification, investigation, and resolution (pp. 188-205). Hershey, PA: IGI Global.
McNamara, D. S., & Kintsch, W. (1996). Learning from text: Effects of prior knowledge and text coherence. Discourse Processes, 22, 247-288.
McNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14, 1-43.
McNamara, D. S., Levinstein, I. B., & Boonthum, C. (2004). iSTART: Interactive strategy trainer for active reading and thinking. Behavioral Research Methods, Instruments, & Computers, 36, 222-233.
McNamara, D. S., & Magliano, J. P. (2009). Self-explanation and metacognition: The dynamics of reading. In J. D. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Handbook of Metacognition in Education (pp. 60-81). Mahwah, NJ: Erlbaum.
McNamara, D. S., & Scott, J. L. (1999). Training Self Explanation and Reading Strategies. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (pp. 1156-1160). Los Angeles, CA: SAGE Publications.
Mitrovic, A., Suraweera, P., Martin, B., & Weerasinghe, A. (2004). DB-suite: Experiences with three intelligent, web-based database tutors. Journal of Interactive Learning Research, 15(4), 409-429.
National Center for Education Statistics (2011). The Nation’s Report Card: Reading 2011 (NCES 2012–457). Institute of Education Sciences, U. S. Department of Education, Washington, D. C.
National Center for Education Statistics. (2015). The Nation’s Report Card: 2015 Mathematics and Reading assessments (NCES Report No. 2015136). Washington, D. C.: National Center for Education Statistics.
O’Brien, E. J., Cook, A. E., & Lorch Jr, R. F. (Eds.). (2015). Inferences during reading. Cambridge University Press.
Ozuru, Y., Best, R., & McNamara, D. S. (2004). Contribution of reading skill to learning from expository texts. In K. Forbus, D. Gentner & T. Regier (Eds.), Proceedings of the 26th Annual Cognitive Science Society (pp. 1071-1076). Mahwah, NJ: Erlbaum.
Proske, A., Roscoe, R. D., & McNamara, D. S. (2014). Game-based practice versus traditional practice in computer-based writing strategy training: Effects on motivation and achievement. Education Technology Research Development, 62, 481-505.
Puntambekar, S., & Hubscher, R. (2005). Tools for scaffolding students in a complex learning environment: What have we gained and what have we missed?. Educational Psychologist, 40(1), 1-12.
Roscoe, R. D., Allen, L. K., Weston, J. L., Crossley, S. A., & McNamara, D. S. (2014). The Writing Pal intelligent tutoring system: Usability testing and development. Computers and Composition, 34, 39-59.
Roscoe, R. D., Brandon, R., Snow, E., & McNamara, D. S. (2013). Game-based writing strategy practice with the Writing Pal. In K. Pytash & R. Ferdig (Eds.), Exploring Technology for Writing and Writing Instruction (pp. 1-20). Hershey, PA: IGI Global.
Roscoe, R. D., & McNamara, D. S. (2013). Writing Pal: Feasibility of an intelligent writing strategy tutor in the high school classroom. Journal of Educational Psychology, 105, 1010-1025.
Roscoe, R. D., Snow, E. L., & McNamara, D. S. (2013). Feedback and revising in an intelligent tutoring system for writing strategies. In K. Yacef et al., (Eds.), Proceedings of the 16th International Conference on Artificial Intelligence in Education (AIED), (pp. 259-268). Heidelberg, Berlin: Springer.
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153-189.
Shute, V. J. & Psotka, J. (1996). Intelligent Tutoring Systems: Past, Present and Future. In D. Jonassen (Ed.), Handbook of Research on Educational Communications and Technology. Scholastic Publications.
Snow, R. E. (1980). Aptitude, learner control, and adaptive instruction. Educational Psychologist, 15(3), 151-158.
Snow, E. L., Jacovina, M. E., Jackson, G. T., & McNamara, D. S. (2016). iSTART-2: A reading comprehension and strategy instruction tutor. In D. S. McNamara & S. A. Crossley (Eds.) Adaptive educational technologies for literacy instruction (pp. 104-121). New York, NY: Taylor & Francis, Routledge.
VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education 16(3): 227-265.
VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197-221.
VanLehn, K., (1988). Student Modeling. In M. C. Polson & J. J. Richardson (Eds.), Foundations of intelligent tutoring systems (pp. 55-78). Hillsdale, NJ: Lawrence Erlbaum Associates.
Vygotsky, L. (1978). Interaction between learning and development. Readings on the development of children, 23(3), 34-41.
Weston, J., Crossley, S. A., & McNamara, D. S. (2010). Towards a computational assessment of freewriting quality. In H. W. Guesgen & C. Murray (Eds.), Proceedings of the 23rd International Florida Artificial Intelligence Research Society (FLAIRS) Conference (pp. 283-288). Menlo Park, CA: The AAAI Press.
Woolf, B. P. (2010). Building Intelligent Interactive Tutors: Learner-Centered Strategies for Revolutionizing E-learning. Oxford, UK: Elsevier.