1322.26 References

The xAPI standard lays the foundation for the interoperable exchange of learning data. xAPI is both a learning technology standard and a web-service specification that requires a web-services application programming interface (API) for describing, recording, and sharing individual or team performance across digital learning systems. The xAPI standard requires the use of a Learning Record Store (LRS), which is the server-side implementation of xAPI. The LRS allows xAPI data to be shared with other systems that require access to this data. Additional information and access to the standard are available on the ADL Initiative's GitHub site.

This Instruction applies to the acquisition of Learning Management Systems (LMSs), Learning Content Management Systems (LCMSs), Student Information Systems (SISs), Learning Record Stores (LRSs), Competency Management Systems (CMSs), and other IT systems used to manage the delivery of training and education content to DoD learners.

When acquiring a new DoD distributed learning system or updating an existing capability, DoD Components shall evaluate the acquisition of different tools, technologies, and systems or training and education programs using the following considerations to determine how this instruction applies.

• Standards Compliance: DoD Components shall develop or acquire distributed learning systems that support the latest versions and editions of the specifications and standards defined in this Instruction to maximize interoperability.
• Data Interoperability: DoD Components shall acquire distributed learning systems that enable the portability of data to other systems, such as those that support human resources, student information management, and training management. Additionally, adherence to the xAPI standard shall be referenced throughout the acquisition process for any type of training and education system. If xAPI adherence cannot be met, the rationale for not utilizing the xAPI standard shall be provided. These include Requests for Information, Sources Sought Notifications, Statements of Work, and other types of requests for solutions.
• Training: DoD Components shall include a requirement to provide comprehensive training as part of any related IT acquisition. Training shall advance users’ knowledge and skill sets on the use and application of the IT asset (e.g., if an LMS is acquired, training shall be provided to teach each group of users how to successfully use the LMS to meet their responsibilities). Component organizations shall determine the best format and source for training. A Continuing Education Unit (CEU) program for such training is recommended.
• LRS Integration and Authentication: This instruction does not define how an LRS is integrated into another IT system. An LRS may be included as part of a single product or may be deployed separately as a separate capability. However, stored data must be sharable and cannot be trapped in the data system. While this document describes the LRS capability using language describing a single system (e.g., “an LRS”), it is likely that multiple LRSs are used to filter, sort, and redirect data. The entire capability is still referred to as “LRS”.

The xAPI Statement Data Model provides content developers with the ability to represent learning experience and performance data in a structured manner. Statements are modeled using JSON Objects and are fundamentally expressed in the form “Actor, Verb, Object” or “A Person(s) did something.” Statements also typically include additional information in the Result and Context Objects to add more meaning or details about the learning experience. A high-level diagram of the primary parts of the xAPI Statement Model is provided below.

Actor will represent the individual person (Actor) who performed the action (Verb) in a given Statement. The Actor in an xAPI Statement SHALL include the following:

• If the Actor is a learner, set the actor.objectType property with the value set to “Agent” unless defined differently in a specific xAPI Profile. If the Actor is a group of learners, set the actor.objectType property with the value set to “Group”. If using a group, determine which practices are best for the use case as opposed to the rest of the Actor guidance below.
• Set the actor.account.homepage property with the value set to an organizationally appropriate and controlled URL.
• Set the actor.account.name property with the Electronic Data Interchange Personal Identifier or PIV associated with the user. (i.e., DoD ID).

Verbs convey the action that occurred in an xAPI Statement. The Verb in an xAPI Statement is represented in the past tense since the Statement is triggered immediately after a learning experience or event occurs.

• The Verb in an xAPI Statement SHALL include the following:
• Set the verb.id to the identifier associated with the relevant Verb.
• Set the verb.display to the human-readable, past tense representation of the Verb.
• The Language Map for the Verb display SHALL include a display string in English with the language code of “en”.

Objects of an xAPI Statement define the subject or object that was acted on. The Object can be an Activity, Agent/Group, Sub statement, or Statement Reference. By default, Objects will be an Activity unless otherwise specified in a particular xAPI Profile. Objects are defined by their Activity Type. Some examples of Activity Types include the following: course, lesson, video, question, page, file, link, etc.

• The Activity Object in an xAPI Statement SHALL include the following:
• The object.id shall reference a unique Activity ID used by the learning activity that generates the xAPI statement, or as a reference to a different activity.
• Set the object.definition.name to the language map value that represents the official name or title of the Activity.
• Set the object.definition.description to the text value that represents a short description of the Activity.
• Set the object.definition.type to the identifier associated with the relevant Activity Type.

• The Activity Object in an xAPI Statement SHALL NOT:
• Use multiple Activity IDs to represent the same Object.
• Reuse the same ID to represent different activities.

These requirements ensure that there is no possibility of accidentally creating and using the same Activity IDs for different activities.

The Activity ID for an Object in an xAPI Statement includes the following requirements:

1. The Activity ID is based on a valid internationalized resource identifier (IRI) starting with https://.
2. The Activity ID SHALL NOT include any spaces.
3. An Activity ID SHALL NOT end with a trailing slash “/” unless the slash is required to resolve to the URL of an external resource.
4. For an Activity that is a link to an external resource (such as an external website), use that resource’s URL as the Activity ID. This requirement only applies to external links.
5. The Activity ID SHALL NOT include a file name extension or the location of a file as part of the ID unless it’s required to resolve to the URL of an external resource.
6. The Activity ID SHALL NOT include any URL-encoded characters unless it’s required to resolve to the URL of an external resource.
7. For all other types of activities, an Activity ID SHALL include a Universally Unique Identifier (UUID) at the end of the IRI to make the Activity ID unique.
8. Do NOT use multiple Activity IDs to represent the same Object or reuse the same ID to represent different activities.
9. Create a unique Activity ID according to the recommended scheme and example table below. Note: The Uniform Resource Identifier (URI) in this example is structured around The Naval Education and Training Command, the hosting organization of the activity. Different DoD Components would use URIs within their control.
10. Content developers SHALL maintain an inventory list of Activity IDs used for each project in order to avoid causing Activity ID collisions by accidentally creating and using the same Activity IDs for different activities. The Activity ID inventory list is a required document that should be updated and shared with all relevant stakeholders.

Context: The Context property of an xAPI Statement contains additional information related to a learning experience. It provides a place to add some contextual information to a Statement. It could store information such as the instructor for an experience, if this experience happened as part of a team-based Activity, or how an experience fits into some broader activity. Please see the general xAPI Statement requirements for examples of these objects and their properties in the sections below.

Context Activities: Many statements do not just involve one (Object) Activity that is the focus but relate to other contextually relevant Activities. The context.ContextActivities property allows for these related Activities to be represented in a structured manner. Valid context types include “parent”, “grouping”, “category”, and “other”.

Context Activities Category: When an xAPI Statement is composed, the ID of the xAPI Profile Activity it conforms to SHALL be declared (using the id property and other optional properties) in the category array as part of the context.ContextActivities property. Additional Profile Activity IDs for each xAPI Profile SHALL also be declared in the category array for each xAPI Profile that is used in an xAPI Statement (e.g., https://w3id.org/xapi/cmi5/context/categories/cmi5).

Context Registration: The context.registration property is used to identify multiple xAPI Statements that are all part of a particular user’s experience throughout interacting with that activity. It should be managed differently than the concept of an attempt (e.g., a user may fail one attempt on an assessment and re-take the assessment as a part of another attempt). The value of the Registration property SHALL be a UUID and should persist throughout all Statements during the registration and likely across all attempts. There is no expectation that completing an Activity ends a registration.

Context Extensions: The values of the context.extension property can be any JSON name/value pair or a JSON Object. Extensions in the "context" property provide context to the core experience, while those in the "result" property provide elements related to some outcome. Within Activities, extensions provide additional information that helps define an Activity within some custom application or Community of Practice.

Context Platform: The context.platform property property is used to specify the platform (e.g., software or hardware) used while the Actor experienced the content. xAPI Statements are required to include the context.platform property property if the value is known. The value of the context.platform property property SHALL be a text string and will vary depending upon the xAPI Profile used.

Timestamps: The timestamp property is used to provide the time when a learning experience occurred. All Statements SHALL include a timestamp. A timestamp SHALL be formatted according to the RFC 33393. This format uses the Gregorian Calendar and Coordinated Universal Time (UTC). The “Z” suffix denotes a UTC offset of 00:00, which is known as Zulu time. For example: 2020-04-30T23:20:50Z. This example represents 20 minutes and 50 seconds after the 23rd hour of April 30th, 2020, in UTC. In contrast, 2020-06-27T12:55:32-0500 represents 55 minutes and 32 seconds after the 12th hour of June 27th, 2020 in the Eastern Time Zone (GMT-5).

• The timestamp in an xAPI Statement SHALL include the following:
• The timestamp must represent the date/time of when the event occurred. Not a future time.
• The timestamp shall be formatted according to RFC 3339 (ISO 8601 normal).
• The timestamp shall be formatted using the Gregorian Calendar with a time zone offset specified.

There are additional best practices to follow for authoring tools, xAPI Profiles, and data design. DoD organizations should use cmi5- or xAPI-supported authoring tools for the creation of xAPI content. If using non-cmi5 xAPI Profiles, the authoring tool should support xAPI version 1.0.3 or later. Context agents were introduced in xAPI 2.0, so version 2.0 should be used if DoD organizations want to take advantage of that addition. The xAPI Statements generated by an authoring tool should be compared to available xAPI Profiles to ensure interoperability. Since xAPI enables many more opportunities for the expression and tracking of learning experiences, reporting on xAPI data generated by distributed learning content can be complicated. When working with xAPI content, the following processes shall be followed:

1. When mapping learner interactions to xAPI Statements, the xAPI implementation (e.g., Verbs, Activity Types, concepts, patterns) should leverage a suitable xAPI Profile. Existing xAPI Profiles shall be used rather than creating a new xAPI Profile that performs the same function. The ADL Initiative maintains a listing of these profiles deemed to be suitable for DoD usage.
2. When mapping learner interactions to xAPI Statements in the context of an LMS or LMS-like activity, first use those in cmi5 to align with the typical Statements found in tracking a learner’s activity in the LMS distributed learning model.
3. If the intended function of an xAPI Verb is slightly different from an existing verb, or additional information is needed, use the xAPI properties such as context, result, or extensions to add this data to the Statement.
4. Where applicable, the use of multiple xAPI Profiles is encouraged. Examples include using cmi5 for course-based content and the Video Profile for any sort of media (audio/video). A complete list of known xAPI Profiles can be accessed from the xAPI Profile Server .
5. If existing xAPI Profiles do not meet requirements, then consider creating a new xAPI Profile using the xAPI Profile Server Authoring Guide located on the ADL Initiative website. All new xAPI Profiles shall be shared with the ADL Initiative such that they may be put online for discovery at a single point of reference.
6. Profile authors SHALL follow Internationalized Resource Identifier (IRI) design best practices. The following IRI pattern should be adopted by anyone creating new concepts for a profile: https://w3id.org/xapi/ [profile name] / [concept type] / [concept]. Profile authors should only customize the content in the IRI in brackets. For example, the Video Profile Verb, https://w3id.org/xapi/video/verbs/seeked, follows this pattern.
7. xAPI Profiles should include information about the profile, such as the name, description, authoring organization or person, and the publication date/time.
8. Those already familiar with xAPI and implementing cmi5 should adhere to the following conformance guidance when designing learning content:
   • Use of specific verbs and results as documented in the cmi5 specification.
   • Use of cmi5-defined Verbs once per user per activity.
   • Use “tagging” statements with contextActivity as defined in the cmi5 specification.

The cmi5 specification, and to-be standard as IEEE 9274.3.1, is the initial step forward for all LMS and non-LMS based content. It enables the packaging and delivery of distributed learning resources that sit inside an LMS, elsewhere within a browser environment, and even outside of a web browser (e.g., mobile apps, simulation content).

• The cmi5 specification has a clear advantage over SCORM, specifically relating to the authenticity of learner data. The use of timestamps and incorporation of the SCORM cmi.interactions model into xAPI provides an advantage in detecting cheating through data manipulation. It also allows the LMS to implement its own sequencing model while providing structure for those LMSs that wish to obtain it from an external source, such as the original course author. The cmi5 specification enables LMS to migrate away from using SCORM, which includes control of the learning environment by an LMS Administrator, which was sometimes automated in SCORM solutions but is an integral part of typical learner management.
• cmi5 “courses” (collections of granular/modular content put together in a Course Structure Format) allow an LMS administrator to edit learner or course details like mastery score if they would like to do so. In this regard, a cmi5 course author is providing recommendations to an LMS administrator that can be used or not upon the importing of that course. Similarly, the completion criteria for a course can be altered by the LMS administrator. This means that cmi5 courses do not need to be “re-authored” to simply change behaviors, but it does mean more intervention and handling will be needed by an LMS administrator.
• When launching content from a cmi5 LMS, the LMS shall be configured to perform the launch using the cmi5 launch protocol. In this case, the content does not need to form the xAPI Actor or hardcode the LRS credentials. This information is provided to the content as part of the launch. The content shall use the information provided via launch over any locally configured information.
• For content that is not web-based or situations where there is a need to have the Actor’s identity anonymized, DoD Components should use LRS Endpoints and localized authentication requirements for launching, testing, and deploying xAPI.
• To enable the migration from SCORM content, which allows for internal behaviors relative to learner performance and to course structure, cmi5 MAY be extended to support the use cases of “testing out” and defining “pre-requisites”.
• Open source tools and templates are available at the following link: https://github.com/adlnet/CATAPULT/ These are free templates but are intended to be starting points for DoD Components. DoD components may develop alternate solutions and should do so as requirements necessitate. This Instruction provides the following recommendations for the use of these resources for cmi5 acquisition. DoD Components pursuing acquisition of cmi5 conformant systems or content SHALL:
• Leverage cmi5 course templates. Modifications may be made to templates to add additional features or to support additional use cases. The use of standardized templates will reduce the time to create new cmi5 content and repurpose existing content. Content may also be modified before or after application of the templates.
• Use the cmi5 Test Suite to verify that cmi5 content, LMSs, and/or authoring tools (that produce cmi5 content) are conformant to the cmi5 specification.
• Test cmi5 content in an environment as close as possible to the end-user environment (cmi5 LMS). If the end-user environment is not available for this purpose, then use the cmi5 Player , such as the open-source player provided by the ADL Initiative, to demonstrate the cmi5 courseware’s functionality.

In the context of distributed learning, metadata provides information about learning content (e.g., author, file size, subject, title, duration). Many types of metadata exist, including descriptive, structural, administrative, reference, and statistical metadata. Metadata describes Learning Resources at any level of granularity and Learning Events, which are opportunities that are contextualized by time and place. Each type has varying characteristics, including those of assets, content, learning resources, and learning activities. Learning metadata can be used to facilitate defense-wide search and discovery of each resource and event, enables artificial intelligence and machine learning, and provides improved insight into the lifecycle of DoD learning resources.

DoD Components should adhere to the following metadata guidance for their learning content as well as other applicable training and education materials:

• All learning resources and where possible, events (e.g., courses, activities, course offerings) should be tagged with metadata. The emerging IEEE P2881 metadata standard should be used when creating learning metadata tags for distributed learning resources and events as it becomes available. The IEEE P2881 standard references and therefore reuses many existing metadata standards.
• Develop a strategy that allows all learning activity metadata to be accessible and interoperable across the DoD. Such solutions allow for storage and retrieval of Learning Resources or the metadata records themselves. Solutions should also provide the ability to update the metadata at a single point (described in the URI recommendations outlined above in section 4.3. Authoring xAPI Enabled Resources - item 6).
• Learning metadata should include data that describes the alignment of Learning Objects to the different educational frameworks used by an organization, such as competency frameworks (e.g., Sharable Competency Definitions), Enabling and Terminal Learning Objectives (ELOs and TLOs), Skill matrices; Training & Evaluation Outlines; or other established frameworks). Alignment metadata should delineate whether the Learning Object teaches, assesses, or both.
• Consider a metadata strategy that delineates each Learning resource (or events) as having specific properties for the diverse types of learning activities used within an organization to educate and train. Learning activity types might include courses, webinars, on-the-job training, virtual classrooms, lectures, or any xAPI activity type. Learning activity types may be embodied within a P2881 application profile. Each P2881 profile should include only fields specific to that learning activity type (e.g., technical information, contextual data, pointers to other data types collected in an activity). Constraints on metadata properties (e.g., allowed terms, value ranges, etc.) should also be documented in metadata application profiles.
• Metadata may include additional technical information, contextual data about the delivery of an instance of that Learning resource or event, and pointers to other types of learner data collected within a particular activity.
• A learning event shall be used to uniquely identify multiple instances of a learning activity/resource. Learning events may describe multiple delivery schedules for the same learning activity, but as uniquely identified objects. A learning instantiation should consider how logistics such as seats, instructors, or time slots are managed in their data strategy.
• Metadata about Learning events should have a location (physical or virtual), start/end dates, and a schedule.
• Metadata should include data about the current version of a Learning Object and should reference one or more active versions of it.
• Metadata should describe the lineage and provenance of different learning activities, establishing revisions, derivations, and representations of those objects using a matrix of Activity IDs.
• Use IRIs (or URIs) for all identifiers, such that they can be transitioned to a semantic web or DoD schema server in the future. Whenever possible, store relevant information at the resolution point of the IRI. This information should include, at minimum, a resource description and location.
• Do not design metadata around specific coding bindings (like XML). DoD Components should define metadata using subject-predicate-object type relationships as seen in semantic web environments using the Resource Description Framework Model and Syntax Specification . Each entity should exist once, and all data should point to that entity.
• Avoid use of complex activity types and “nested” data elements within metadata (such as defining an author of a course and then putting author name and contact information inside the author “tag”).

To enable a competency-based / outcome-based learning strategy, DoD Components shall begin to define competency definitions, competency associations, and competency frameworks as described in IEEE 1484.20.3 –SCD. The granular unit, often referred to as a “competency”, is a competency definition in SCD. These are referred to as “competencies” throughout this Reference.

Based on industry best practices and the IEEE 1484.20.3 draft standard, DoD Components should:

• Implement a strategy that allows all competencies to be accessible and interoperable across DoD Components. Competency Definitions and Competency Frameworks should be referenceable within a Competency Registry that allows for storage and retrieval of competencies and their associated Frameworks. Competencies and their Frameworks should also support versioning and alignment of similar competencies. Competency Registries should be able to update competencies and their frameworks at a single, referenceable, IRI (following the IRI recommendations outlined below).
• Document local competencies in a manner that adequately describes each competency and meets the following requirements:
• Competency Definitions shall include unique IDs in the form of an IRI. Each competency should also have a text representation that is definitive and descriptive.
• Competency Definitions shall include a human-readable expression that describes a competency (e.g., Operation of a Gear Stick and Clutch in a Commercial Vehicle)
• Competency Definitions shall include a narrative in plain language that describes and contextualizes the competency (e.g., This competency relates to the operation of a commercial vehicle that a civilian would normally operate. Certain vehicles have a manual transmission, which requires the ability to manually control a gear stick with precision and timing as the vehicle changes gears when moving between drive/neutral/park and when reaching certain speed thresholds.)
• Competency Definitions shall include a name or label that would be viewed within a competency framework. (e.g., ManualTransmissionMastery)
• Competency Definitions and Competency Frameworks should support resource association that describe relationships within the context of the competency framework and its ecosystem. Competency Frameworks can be created at multiple levels. An industry best practices guide for competency-based education has also been published by IEEE (1484.20.2) that should be used to inform DoD implementations.
• Relationships (expressions) between Competency Definitions should be described using an association property. The types of relationships should focus on both hierarchical (one member belongs to a larger group) and ordered (an intended pre-requisite exists) relationships. This overrides any generic notion of parent/child relationships of competencies; the design should be with intent. Examples would be a set of quiz questions making up an assessment (hierarchical) or a direct series of lessons that build on the previous lesson making up a course (ordered).
• Use an association property to link to other relevant organizational Competency Frameworks, as applicable.
• Competency Definitions should include assessment rubrics to describe performance criteria for different proficiency levels within an individual competency where applicable, Assessment rubrics should separate one level of competence from another. Creation of rubrics is recommended. Levels may be designed using the rubric criterion if implemented within a rubric.
• When versioning competencies and Competency Frameworks, maintain the version history, including if a competency or competency framework was derived from a separate effort that is not considered a previous version.
• Consider using IEEE 1484.20.3 SCD extensions to describe relationships and metadata within competencies and competency frameworks using public schemas.

To better align learner data with human capital management systems and software systems used across other DoD functional communities, DoD Components should start to define and document occupations/jobs, roles, tasks, learning opportunities, credentials, and assessment profiles in a digital representation. The preferred mechanism to do this is Credential Transparency Description Language (CTDL). The scope of the CTDL is restricted to a description of credentials offered and not the description of credentials awarded. In other words, awarding, appraising, or validating a credential, including the information about the person earning the credential and the issuing authority and the dates of associated events are not in scope of CTDL.

CTDL is used to align competencies and credentials with jobs, career milestones, and future opportunities when transitioning from DoD into the civilian sector.

CTDL relies on linked data to facilitate semantic interoperability as an approach to combining data about things from different sources, particularly among the vocabularies used to define credentials, competencies, jobs, tasks, etc.

The CTDL family of specifications is built on the principles of the Resource Description Framework (RDF) approach to describing resources. As such, CTDL allows for a linked data approach to combining information about resources from different sources, the use of CTDL with terms from other RDF specifications and expressing the relationship between terms in CTDL to similar terms in other specifications. In some cases, CTDL has been designed in the expectation that there will be future integration. In some cases, a standards development organization (e.g., IEEE) may overlap with CTDL. If a data conflict arises, choose the solution that uses the standards development organization.

The W3C Recommendation on Verifiable Credentials (W3C-VC) is complementary to CTDL in that the W3V VC data model allows the expression of the information needed to appraise and validate a credential held by an individual. The Recommendation also describes how such data can be made tamper-evident and can be shared within a distributed credentialing ecosystem. The terms in the data model alone do not allow for much information about the nature of the credential, which is deliberately left as broad as possible, however for those credentials that relate to educational or occupational achievements CTDL can be used to fill in the details.

In the future, CTDL may include integration to other standards, such as IEEE’s SCD and define pathways to competency frameworks. Currently, CTDL leverages its own rules that are specific to competencies and competency frameworks, but these are not recommended at this time.

If a DoD Component chooses to implement competency-based learning, SCD is still the preferred strategy. Should there be a conflict between details of SCD and CTDL, CTDL should not be used.