Presentation
Joint Activity Testing: Towards a Multi-Dimensional, High-Resolution Evaluation Method for Human-Machine Teaming
Event Type
Poster
In-Person
TimeWednesday, October 6th5:00pm - 6:00pm EDT
LocationHarborside CDE
DescriptionQuantitative evaluations of human-machine teams (HMTs) are desperately needed to ensure technological implementations are helpful rather than harmful to overall system performance; however, as machines increasingly behave like active cognitive teammates, traditional evaluation strategies risk overestimating HMT capabilities. A reliable HMT evaluation method should include multiple high-resolution, continuous measures for both system performance and system challenges that can be implemented unobtrusively in real-time operations. In our prior work, we proposed joint activity testing (JAT) as a candidate evaluation framework to satisfy these requirements. Preliminary efforts with a single dimension of performance and challenge have indicated that the method can identify the additive benefits of joint activity with a specific technology. In this paper, we explore the operationalization of multi-dimensional JAT by synthesizing our work in two intelligence and two healthcare domains. The patterns observed between domains will guide future JAT, reveal paths towards real-time implementation, and spark future research evaluating resilience.
