Using Competencies to Map Performance across Multiple Activities

2017; IITSEC; Robson, R.; Poltrack, J.

When a single training system accumulates data on learner performance, the data are stored in a way determined by the system's designers. This enables the system to access these data and to apply them to its interactions with learners.In environments such as live-virtual-constructive federations, each component may store performance data in its own way, making it difficult for one component to access and use data produced by another. To enable cross-component sharing of performance data,it is necessary to establish shared definitions of skills and outcomes; create a common language for expressing performance data;interpret data produced at wildly differing levels of granularity; and (in some cases) satisfy a large array of security and privacy requirements. This paper is based on work done by the US Advanced Distributed Learning (ADL) Initiative, the Credential Engine foundation, and several standards bodies. It starts by discussing the above challenges and their manifestations in use cases ranging from federations of sophisticated adaptive training and simulation systems to more traditional online learning environments. The paper then describes a potential solution for collecting and processing assertions of competency, skills, and performance from multiple sources. Each assertion is of the form "Learner X has (or has not) achieved competency Y at level Z with confidence p based on evidence E." "Competencies" are drawn from shared, machine-readable frameworks that can represent knowledge, skills, ability, and objectives. Assertions can be collected directly or generated by ingesting granular performance data and correlating it to competencies, enabling algorithms that use explicit rules and relationships to draw further inferences. We have recently tested one such system with Army special operators (n = 79) at the JFK Special Warfare Center and School as part of the ADL's Total Learning Architecture demonstration. The paper discusses our approach to establishing trusted networks and complying with privacy and security requirements; how we used the Experience API (xAPI) and industry standards; and lessons learned from that event.

Citation: Poltrack, J., Robson, R. (2017). Using competencies to map performance across multiple activities [Paper presentation]. In Volume 2017 (Eds.), Proceedings of the 2017 Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC). Arlington, VA: National Training and Simulation Association.

Download Publication