The combination of different sensory information to predict upcoming situations is an innate capability of intelligent beings. Consequently, various studies in the Artificial Intelligence field are currently being conducted to transfer this ability to artificial systems. Autonomous vehicles can particularly benefit from the combination of multi-modal information from the different sensors of the agent. This paper proposes a method for video-frame prediction that leverages odometric data. It can then serve as a basis for anomaly detection. A Dynamic Bayesian Network framework is adopted, combined with the use of Deep Learning methods to learn an appropriate latent space. First, a Markov Jump Particle Filter is built over the odometric data. This odometry model comprises a set of clusters. As a second step, the video model is learned. It is composed of a Kalman Variational Autoencoder modified to leverage the odometry clusters for focusing its learning attention on features related to the dynamic tasks that the vehicle is performing. We call the obtained overall model Cluster-Guided Kalman Variational Autoencoder. Evaluation is conducted using data from a car moving in a closed environment [1] and leveraging a part of the University of Alcalá DriveSet dataset [2], where several drivers move in a normal and drowsy way along a secondary road.

A Kalman Variational Autoencoder Model assisted by Odometric Clustering for Video Frame Prediction and Anomaly Detection

Giulia Slavic;Abrham Shiferaw Alemaw;Lucio Marcenaro;Carlo Regazzoni
2022-01-01

Abstract

The combination of different sensory information to predict upcoming situations is an innate capability of intelligent beings. Consequently, various studies in the Artificial Intelligence field are currently being conducted to transfer this ability to artificial systems. Autonomous vehicles can particularly benefit from the combination of multi-modal information from the different sensors of the agent. This paper proposes a method for video-frame prediction that leverages odometric data. It can then serve as a basis for anomaly detection. A Dynamic Bayesian Network framework is adopted, combined with the use of Deep Learning methods to learn an appropriate latent space. First, a Markov Jump Particle Filter is built over the odometric data. This odometry model comprises a set of clusters. As a second step, the video model is learned. It is composed of a Kalman Variational Autoencoder modified to leverage the odometry clusters for focusing its learning attention on features related to the dynamic tasks that the vehicle is performing. We call the obtained overall model Cluster-Guided Kalman Variational Autoencoder. Evaluation is conducted using data from a car moving in a closed environment [1] and leveraging a part of the University of Alcalá DriveSet dataset [2], where several drivers move in a normal and drowsy way along a secondary road.
File in questo prodotto:
File Dimensione Formato  
doubleColumn_with_ACK.pdf

accesso chiuso

Descrizione: Articolo su rivista
Tipologia: Documento in Post-print
Dimensione 6.7 MB
Formato Adobe PDF
6.7 MB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11567/1102877
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact