Mathematical Methods for eXplainable and Reliable Machine Learning in Trustworthy Artificial Intelligence

The common thread in all my research has been the goal of finding a so-called safety region in the input space of an inference model that allows providing probabilistic guarantees on the output of the model and tools to control the prediction. The idea of safety region fits well with the task of classification in machine learning: the goal is to classify instances into well-defined and closed envelopes, respecting some probabilistic performance or guarantees. So my research started from a thorough and accurate review of the main classification algorithms in machine learning, from support vector machines to neural networks via rule-based models as well. But the best algorithm I found to achieve my purpose was Support Vector Data Descrip- tion (SVDD), an established algorithm for outlier detection whose main purpose is to enclose target data within a sphere with a center and radius learned from the data distribution. The choice of such an algorithm for defining the safety re- gion is quite trivial and supportable: SVDD allows a closed region to be defined in the input space and also provides a radius that can easily control the shape of the classification boundary to “inflate” or “deflate” it according to the performance objective. Starting from a totally data-driven definition of safety region, with only empirical (but effective) performance guarantees, I moved to a more mathematical definition, placing my idea of safety region within the framework of probabilistic scaling. This technique, in the state of the art of order statistics, provides a clear and indisputable way to obtain probabilistic guarantees on the safety region. Here, moreover, I applied the idea of safety region to a broader class of classifiers, called scalable classifiers, i.e., classification models that all share a scalable parameter in the classifier’s predictor definition that can be appropriately adjusted to obtain the desired guarantees for the safety region and I also specialized these concepts into exponential distributions that allow special properties of safety regions. This allows to extend the concepts developed in Chapter 3 from SVDD to any kind of ma- chine learning classifier. In particular, I introduced new algorithms both to control performance in classification and to obtain probabilistic guarantees of the safety region. Performance control was achieved by minimizing the misclassification error, reducing the number of false positives or false negatives or both, depending on the application. On the other hand, probabilistic guarantee has been shown mathemat- ically to be effective. Both concepts, however, can be applied to real-world problems to achieve safety in cyber-physical systems applications, such as vehicle platooning monitoring, DNS tunneling detection, and type-2 diabetes disease prediction, just to name a few tested applications of my methods. However, before getting good results in my research, several ways were tried. An- other line of research for defining the safety region was the use of conformal predic- tion, a new but well-established theory for evaluating conformity in machine learning algorithm performance. In this case, the idea behind conformal prediction is that it is possible to correctly calibrate an algorithm to obtain marginal probability cover- age that the desired output of the model is as expected. In this field, it is necessary to define a real-valued function, called score function, that encodes the characteris- tics of the model and calibrate the algorithm to the result of evaluating that function on a calibration set. This line of research is getting good prospects and is one of the lines I will follow in my future work. But reliability is not enough to make AI totally trustworthy. In fact, controllability is another crucial aspect to consider. From this point of view, I focused on studying and developing new techniques to control the output of a classification algorithm. This was done in the spirit of counterfactual explanation, a fairly new but already state-of-the-art eXplanaible AI technique. The idea of counterfactual explanations is that it is possible to minimally change the input parameters of a machine learning algorithm so as to change the prediction results. In the sense that will be explained in the chapter dedicated to counterfactual explanations (Chapter 9) will be clear that the expression “ minimal change” refers to the idea of minimizing a specific cost function between the actual input and the desired one. My contribution in this topic lies in the development of a counterfactual approach based on SVDD, totally in line with the idea of safety region investigated in the first part of my research. The proposed approach was first attempted to be solved completely analytically, but then, given the complexity of the task, a numerical solution based on random sampling techniques was developed. The algorithm, again, was applied to real-world application problems, such as crowd control in subways. This topic, however, allows for more exploration, for example by merging it together with the conformal frame- work provided by safety regions. Finally, all the work presented in this thesis has been surrounded by explainable AI, the field of study dedicated to making AI explainable and expressible by intelligible rules. In this regard, explainable AI can also be declined in terms of controllability and reliability, thus placing all my research totally in line with this theme. In conclusion, my thesis covered three years of research in the field of artificial intelligence, spending most of the time evaluating the problem of how to make a good machine learning algorithm from a reliable, explainable and controllable point of view, with the hope of having really improved the body of knowledge in such a crucial aspect of Science.