CEST: a Cognitive Event based Semi-automatic Technique for behavior segmentation

This work introduces CEST, a Cognitive Event based Semiautomatic Technique for behavior segmentation. The technique was inspired by an everyday cognitive process. Humans, in fact, make sense of what happens to them by breaking the continuous stream of activity into smaller units, through a process known as segmentation. A cognitive theory, the Event Segmentation Theory, provides a computational and neurophysiological account of this process, describing how the detection of changes in the current situation drive boundary perception. CEST was designed with the aim of providing affective researchers with a tool to semi-automatically segment behavior. Researchers investigating behavior, as a matter of fact, often need to parse their research data into simpler units, either manually or automatically. To perform segmentation, the technique combines manual annotations and the output of change-point detection algorithms, techniques from time-series research that afford the detection of abrupt changes in time-series. CEST is inherently multidisciplinary: it is, to the best of our knowledge, the first attempt to adopt a cognitive science perspective on the issue of (semi) automatic behavior segmentation. CEST is a general-purpose technique, as it aims at providing a tool for segmenting behavior across research areas. In this manuscript, we detail the theories behind the design of CEST and the results of two experimental studies aimed at assessing the feasibility of the approach on both single and group scenarios. Most importantly, we present the results of the evaluation of CEST on a data-set of dance performances. We explore seven different techniques for change-point detection that could be leveraged to achieve semi-automatic segmentation through CEST and illustrate how two different bayesian algorithms led to the highest scores. Upon selecting the best algorithms, we measured the effect of the temporal grain of the analysis on the performance. Overall, our results support the idea of a semiautomatic segmentation technique for behavior segmentation. The output of the analysis mirrors cognitive science research on segmentation and on event structure perception. The work also tackles new challenges that may arise from our approach.