Framework for the Design of Accessible Intelligent Tutoring Systems

Eric G. Hansen, Diego Zapata-Rivera, & Jason White

Corresponding Author: dzapata@ets.org

Abstract

We describe a framework for designing intelligent tutoring systems (ITSs) that are accessible to learners with disabilities. This framework is based on evidence centered design (ECD), which treats assessment designs, and their associated pedagogy, as parts of an evidentiary argument. That evidentiary argument involves claims about what a learner knows and can do and evidence for those claims. In the framework we specify three levels of ITS accessibility (basic, enhanced, and self-adaptive) that build upon each other with respect to their adaptivity to the needs of learners with disabilities. In the chapter we describe the framework and its application to ITSs and suggest areas for further research and development, including the need to develop ECD-based tools and improved technical standards.

Keywords: accessibility, assessment, disabilities, evidence centered design, intelligent tutoring systems, English language learners

Back to Table of Contents

Link to Book

APA citation information

Hansen, H. G., Zapata-Rivera, D., & White, J. (2018). Framework for the design of accessible intelligent tutoring systems. In S. D. Craig (Ed.). Tutoring and Intelligent Tutoring Systems (pp. 69-101). New York, NY: Nova Science Publishers.

References

36 CFR Part 1194.1 (2017). Information and communication technology standards and guidelines. Retrieved from www.law.cornell.edu/cfr/text/36/1194.1.

36 CFR Section 1194.2 (2017). Access to advance communications services and equipment by people with disabilities. Retrieved from www.law.cornell.edu/ cfr/text/36/1194.2.

American Printing House for the Blind. (n.d.) Introducing Graphiti — A revolution in accessing digital tactile graphics and more! Retrieved from www.aph.org/graphiti/

Center for Adaptive Instructional Sciences (CAIS). (2018). Standards abstracts. Retrieved from instructionalsciences.org/projects/cais/boards.

Chughtai, R., Zhang, S., & Craig, S. D. (2015, September). Usability evaluation of intelligent tutoring system: ITS from a usability perspective. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 59, No. 1, pp. 367-371). Los Angeles, CA: SAGE.

Clark, R. C., Nguyen, F., & Sweller, J. (2011). Efficiency in learning: Evidence-based guidelines to manage cognitive load. San Francisco, CA: Wiley.

Common Core State Standards Initiative. (2010). Common Core State Standards for Mathematics. Washington, DC: National Governors Association Center for Best Practices and the Council of Chief State School Officers.

ETSI (2014). EN 301 549 V1.1.1, Accessibility requirements suitable for public procurement of ICT products and services in Europe. www.etsi.org/deliver/ etsi_en/301500_301599/301549/01.01.01_60/en_301549v010101p.pdf.

Feng, M., Hansen, E. G., & Zapata-Rivera, D. (2009, April). Using evidence centered design for learning (ECDL) to examine the ASSISTments system. Paper presented at the meeting of the American Educational Research Association, San Diego, California.

Generalized Intelligent Framework for Tutoring (GIFT). (2018). Description. Retrieved from gifttutoring.org/projects/gift/wiki/Overview#Description.

Global Public Inclusive Infrastructure (n.d.). Global Public Inclusive Infrastructure. Retrieved from gpii.net.

Graesser, A. C., Hu, X., Nye, B., & Sottilare, R. (2016). Intelligent tutoring systems, serious games, and the Generalized Intelligent Framework for Tutoring (GIFT). In H. F., O’Neil, E. L. Baker, and R. S. Perez,(Eds.).Using games and simulations for teaching and assessment: Key issues. (pp. 58-79). New York, NY: Routledge.

Haertel, G., Haydel DeBarger, A., Cheng, B., Blackorby, J., Javitz, H., Ructtinger, L., … Hansen, E. G. (2010). Using evidence-centered design and universal design for learning to design science assessment tasks for students with disabilities (Assessment for Students with Disabilities Technical Report 1). Menlo Park, CA: SRI International.

Hansen, E. G. (2011). Evidence-centered design for learning (Research Memorandum 11-02). Princeton, NJ: Educational Testing Service.

Hansen, E. G., & Mislevy, R. J. (2005). Accessibility of computer-based testing for individuals with disabilities and English language learners within a validity framework. In M. Hricko & S. Howell (Eds.), Online assessment and measurement: Foundation, challenges, and issues (pp. 214-262), Hershey, PA: Idea Group Publishing.

Hansen, E. G., Mislevy, R. J., Steinberg, L. S., Lee, M. J., & Forer, D. C. (2005). Accessibility of tests for individuals with disabilities within a validity framework. System: An International Journal of Educational Technology and Applied Linguistics, 33, 107-133. doi: doi.org/10.1016/j.system.2004.11.002.

Hansen, E. G., Shute, V. J., & Landau, S. (2010). An assessment-for-learning system in mathematics for individuals with visual impairments. Journal of Visual Impairment and Blindness, 104, 275-286.

Hansen, E. G., & Zapata-Rivera, D. (2010a, May). Evidence centered design and cognitive load theory for improving the accessibility and learning efficiency of assessment-for-learning systems. Paper presented at the meeting of the American Educational Research Association, Denver, CO.

Hansen, E. G., & Zapata-Rivera, D. (2010b, May). Evidence centered design for an accessible game-based assessment-for-learning system for middle school mathematics. Paper presented at the meeting of the National Council on Measurement in Education, Denver, CO.

Hartley, J. R., & Sleeman, D. H. (1973). Towards more intelligent teaching systems. International Journal of Man-Machine Studies, 2, 215-236.

IMS Global Learning Consortium. (2004). IMS AccessForAll meta-data overview (version 1.0 final specification). Retrieved from www.imsglobal.org/accessibility/ accmdv1p0/imsaccmd_oviewv1p0.html.

ISO/IEC 24751-1 (2008). Preview: Information technology—Individualized adaptability and accessibility in e-learning, education and training—Part 1: Framework and reference model. Retrieved from www.iso.org/standard/41521.html.

Johnson, W. L., & Valente, A. (2008). Tactical language and culture training systems: Using artificial intelligence to teach foreign languages and cultures. AI Magazine, 30(2), 72-83. doi: doi.org/10.1609/aimag.v30i2.2240.

Katz, I. R., LaMar, M. M., Spain, R., Zapata-Rivera, J. D., Baird, J-A., & Greiff, S. (2017) Validity issues and concerns for technology-based performance assessments, In R. A. Sottilare, A. C. Graesser, X. Hu, & G. A. Goodwin (Eds.), Design Recommendations for Intelligent Tutoring Systems: Volume 5 – Assessment Methods, 209-224.

Kirschner, P., Sweller, J., & Clark, R. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41, 75–86.

Lajoie, S. P., & Derry, S. (1993). Computer environments as cognitive tools for enhancing learning. Computers as Cognitive Tools. Hillsdale, NJ: Erlbaum.

Loitsch, C., Weber, G., Kaklanis, N., Votis, K., & Tzovaras, D. (2017). A knowledge-based approach to user interface adaptation from preferences and for special needs. User Modeling and User-Adapted Interaction, 27(3-5), 445-491.

Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13-23.

Millis, K., Forsyth, C., Butler, H., Wallace, P., Graesser, A. C. & Halpern, D. (2011). Operation ARIES! A serious game for teaching scientific inquiry. In M. Ma, A. Oikonomou, & J. Lakhmi (Eds.), Serious games and edutainment applications (pp.169–196). London, UK: Springer-Verlag.

Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to evidence‐entered design. (Research Report 03-1). Princeton, NJ: Educational Testing Service.

Mislevy, R.J. & Gitomer, D.H. (1996). The role of probability-based inference in an intelligent tutoring system. User-Mediated and User-Adapted Interaction, 5, 253–282.

Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1, 3-67.

Mislevy, R. J., & Yan, D. (2017). Evidence-centered assessment design and probability-based inference to support the Generalized Intelligent Framework for Tutoring (GIFT). In R. A. Sottilare, A. C. Graesser, X. Hu, & G. A. Goodwin (Eds.), Design Recommendations for Intelligent Tutoring Systems: Volume 5 – Assessment Methods, 101-123.

National Center on Educational Statistics (2017). Children and youth with disabilities. Retrieved from nces.ed.gov/programs/coe/indicator_cgg.asp.

Razzaq, L., Feng, M., Heffernan, N., Koedinger, K., Junker, B., Nuzzo-Jones, G.. . . . Walonoski, J. (2007). A web-based authoring tool for intelligent tutors: Blending assessment and instructional assistance. In: N. Nedjah, L. M. Mourelle, M. N. Borges, N. N. Almeida (Eds.) Intelligent Educational Machines, Studies in Computational Intelligence, 44, 23-49. Berlin, Germany: Springer.

Regian, J. W., & Shute, V. J. (Eds.). (1992). Cognitive approaches to automated instruction. Hillsdale, NJ: Erlbaum.

Roscoe, R. D., Cooke, N. J., Branaghan, R. J., & Craig, S. D. (2017). Human systems engineering and educational technology. In End-User Considerations in Educational Technology Design (pp. 1-34). IGI Global. DOI: 10.4018/978-1-5225-2639-1.ch001

Shute, V. J., Hansen, E. G., & Almond, R. G. (2008). You can’t fatten a hog by weighing it – or can you? Evaluating an assessment for learning system called ACED. International Journal of Artificial Intelligence in Education, 18, 289-316.

Shute, V. J., Graf, E. A., & Hansen, E. G. (2006). Designing adaptive, diagnostic math assessments for individuals with and without visual disabilities (Research Report 06-01) Princeton, NJ: Educational Testing Service.

Shute, V. J., & Zapata‐Rivera, D. (2007). Adaptive technologies (Research Report 07-01). Princeton, NJ: Educational Testing Service.

Sottilare, R. A., Brawner, K. W., Goldberg, B. S., & Holden, H. K. (2012). The generalized intelligent framework for tutoring (GIFT). Orlando, FL: US Army Research Laboratory, Human Research & Engineering Directorate (ARL-HRED).

Sottilare, R., Graesser, A., Hu, X., & Goodwin, G. (Eds.). (2017). Design recommendations for intelligent tutoring system: Assessment methods (Vol. 5). US Army Research Laboratory.

Touch Graphics. (2018). Talking Tactile Tablet 2 (TTT). Retrieved from http:// touchgraphics.com/portfolio/ttt/

World Wide Web Consortium. (2008). Web content accessibility guidelines (WCAG) 2.0. Retrieved from www.w3.org/TR/WCAG20/

Yarnall, L., Snow, E., Snow, E., & Katz, I. R. (2017). Leveraging domain models for personalizing problem solving and learning. In R. A. Sottilare, A. C. Graesser, X. Hu, & G. A. Goodwin (Eds.), Design Recommendations for Intelligent Tutoring Systems: Volume 5 – Assessment Methods, 69-83.

Zapata-Rivera, D., VanWinkle, W., Doyle, B., Buteux, A., & Bauer, M. (2009). Combining learning and assessment in assessment-based gaming environments: A case study from a New York City school. Interactive Technology and Smart Education, 6(3), 173-188.

Zhang, T., Mislevy, R. J., Haertel, G., Javitz, H., Murray, E., Gravel, J., & Hansen, E. G. (2010). A design pattern for a spelling assessment for students with disabilities (Assessment for Students with Disabilities Technical Report 2). Menlo Park, CA: SRI International. Retrieved from padi-se.sri.com/downloads/TR2_SpellingBee.pdf.