Exploring Assessment Mechanisms in the Total Learning Architecture (TLA)

2017; Chapter in Book - GIFT; Goodwin, G; Folsom-Kovarik, J.T.; Johsnon, A.; Schatz, S., Ph.D.; Sottilare, R.

The focus of this chapter is on the challenges and potential solutions to conducting realtime and long-term assessments of performance, learning, and domain competency in the Total Learning Architecture (TLA). TLA, a distributed learning ecosystem, is being developed by the US Office of the Secretary of Defense to support capabilities for instruction anytime and anywhere. TLA is an evolving set of standardized specifications that enable responsible sharing of essential learning data between applications using common interfaces and data models. The applications that could be part of the TLA ecosystem range from simple desktop applications to immersive simulations to mobile apps, and would serve as either service providers or consumers. Expected services include applications like intelligent tutoring systems (ITSs; e.g., Auto Tutor, Cognitive Tutor, or Generalized Intelligent Framework for Tutoring [GIFT]-based tutor), which provide information to other services and consume information from other services. The TLA is expected to provide services including experience tracking, competency assessment, learner modeling, and content brokering. All of these fundamentally involve learner assessments. Experience tracking (via the experience application programming interface [xAPI]) provides a standard for encoding and storing data about learners'interactions with learning experiences and applications, providing fine grained evidence that can make assessment precise and timely. TLA will also establish a common way for systems to reference and represent competencies and competency relationships, supporting assessment sharing. Learner models will contain data about assessed mastery of competencies as well as traits, preferences, individual differences, and demographic data. Learner models that are broadly accessible to learning applications will support uptodate and accurate competency assessment. Content brokering (i.e., recommending future experiences and training) also depends on learner assessments. Content brokering will support just in time learning and sequencing of learning events. Competency models will enable content to be tailored to the individual learner's needs. Training applications like GIFT operating in the TLA environment will both consume learner data available through TLA and provide learner data as they complete training. It will be challenging to insure that all training applications will be able to both obtain necessary learner data from TLA as well as insure that they all output learner measures that can be used by other applications within TLA. This chapter explores some of the challenges of integrating a training application like GIFT into the TLA. This includes discussion of the discovery and development of methods to assess competency based on xAPI statements and recommendations for augmenting xAPI statements to facilitate interoperability among training applications and the TLA through methods such as semantic analysis.

Download Publication