Ego things: Networks Of Self-Aware Intelligent Objects

There is an increasing demand for developing intelligence and awareness in artificial agents in recent days to improve autonomy, robustness, and scalability, and it has been investigated in various research fields such as machine learning, robotics, software engineering, etc. Moreover, it is crucial to model such an agent's interaction with the surrounding environment and other agents to represent collaborative tasks. In this thesis, we have proposed several approaches to developing multi-modal self-awareness in agents and multi-modal collective awareness (CA) for multiple networked intelligent agents by focusing on the functionality to detect abnormal situations. The first part of the thesis is proposed a novel approach to build self-awareness in dynamic agents to detect abnormalities based on multi-sensory data and feature selection. By considering several sensory data features, learned multiple inference models and facilitated obtaining the most distinct features for predicting future instances and detecting possible abnormalities. The proposed method can select the optimal set features to be shared in networking operations such that state prediction, decision-making, and abnormality detection processes are favored. In the second part, proposed different approaches for developing collective awareness in an agent's network. Each agent of a network is considered an Internet of Things (IoT) node equipped with machine learning capabilities. The collective awareness aims to provide the network with updated causal knowledge of the state of execution of actions of each node performing a joint task, with particular attention to anomalies that can arise. Data-driven dynamic Bayesian models learned from multi-sensory data recorded during the normal realization of a joint task (agent network experience) are used for distributed state estimation of agents and detection of abnormalities. Moreover, the effects of networking protocols and communications in the estimation of state and abnormalities are analyzed. Finally, the abnormality estimation is performed at the model's different abstraction levels and explained the models' interpretability. In this work, interpretability is the capability to use anomaly data to modify the model to make inferences accurately in the future.