Performance Measurement in LVC Distributed Simulations: Lessons from OBW

2017; IITSEC; Hoke, J.; Townsend, L.; Giambarberee, S.; Schatz, S., Ph.D.

Operation Blended Warrior (OBW) 2016 marked the second year of a three-year effort to document lessons learned and understand barriers to implementing Live, Virtual, Constructive (LVC) distributed training. In the first year of the event, LVC focus areas included connectivity, interoperability, data standards, after-action review, and cyber security. Year two introduced additional focus areas: multi-level security, cross domain solutions, long-haul feeds, and performance measurement. This paper focuses on this latter area - defining and collecting performance measures. Performance measurement in simulation-based training faces formidable obstacles, including the identification of individual and collective performance dimensions, how these dimensions relate to training goals, and how training transfers to operational readiness. Blending of LVC elements introduces additional complexity, not only for human performance assessment but also for evaluating the effectiveness and efficiency of the technical system. In this paper, we present the measures defined and collected during OBW in four primary areas: 1) cost analysis, 2) network performance, 3) trainee performance, and 4) whether OBW met the expectations of participating organizations. We also discuss three categories of Measures of Effectiveness (MoEs) and Measures of Performance (MoPs) established by the OBW Strategic Integrated Product Team: Programmatic, Technological, and Learning.These MoEs and MoPs will facilitate annual comparisons of performance measurement at OBW and encourage use of the event as a sandbox to design and validate LVC performance measurement tools. Finally, we present the goals and measures established for Performance Measurement during OBW 2017 and recommendations for future events

Citation: Giambarberee, S., Hoke, J., Schatz, S., Townsend, L. (2017). Performance measurement in LCV distributed simulations: Lessons from OBW [Paper presentation]. In Volume 2017 (Eds.), Proceedings of the 2017 Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC). Arlington, VA: National Training and Simulation Association.