This paper proposes a method to localize a moving agent - such as an autonomous surveillance vehicle - inside a known environment using First Person Viewpoint video data. Anomalies w.r.t. expected vehicle motion and image content are extracted to guide the localization, signal when the localization results are not trustworthy, and explain the reason for the failure. During the training phase, a Dynamic Bayesian Network model is learned, which couples positional and video data. To learn it, clustering is performed on the odometry data, and a modified Kalman Variational Autoencoder is built over the video data. During the testing phase, a Coupled Markov Jump Particle Filter leverages the learned Dynamic Bayesian Network to extract anomalies and to estimate the vehicle’s position, given only camera data. The proposed method is evaluated on two real-world datasets of a vehicle performing perimeter monitoring of a closed environment [17] and of a shopping cart moving in a supermarket

Simultaneous Localization and Anomaly Detection from First-Person Video Data through a Coupled Dynamic Bayesian Network Model

Giulia Slavic;Lucio Marcenaro;Carlo Regazzoni
2022-01-01

Abstract

This paper proposes a method to localize a moving agent - such as an autonomous surveillance vehicle - inside a known environment using First Person Viewpoint video data. Anomalies w.r.t. expected vehicle motion and image content are extracted to guide the localization, signal when the localization results are not trustworthy, and explain the reason for the failure. During the training phase, a Dynamic Bayesian Network model is learned, which couples positional and video data. To learn it, clustering is performed on the odometry data, and a modified Kalman Variational Autoencoder is built over the video data. During the testing phase, a Coupled Markov Jump Particle Filter leverages the learned Dynamic Bayesian Network to extract anomalies and to estimate the vehicle’s position, given only camera data. The proposed method is evaluated on two real-world datasets of a vehicle performing perimeter monitoring of a closed environment [17] and of a shopping cart moving in a supermarket
File in questo prodotto:
File Dimensione Formato  
AVSS_2022.pdf

accesso chiuso

Descrizione: Contributo in atti di convegno
Tipologia: Documento in Post-print
Dimensione 3.36 MB
Formato Adobe PDF
3.36 MB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11567/1102876
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact