Notes on an Entropy-like Method for Robust Parameter Identification

This reports describes the basic ideas behind a novel parameter identification algorithm exhibiting high robustness with respect to outlying data. The algorithm consists in minimizing an entropy-like cost function of the identification residuals. Robustness to outliers is obtained as a consequence of the fact that the used cost function rewards unevenly distributed residuals rather than some kind of weighted mean square error (MSE). In particular residuals are normalized to the MSE and the minimization of the devised entropy-like function rewards the presence of a majority of low relative errors and a minority of large ones, rather than a least MSE that tends to level out residuals hence often hiding outliers. A preliminary theoretical analysis of the algorithm is reported together with experimental results to demonstrate the method. This work builds on ideas initially elaborated in 1999 and extended only in the Summer of 2006 during a research visit at the IUB International University Bremen (now Jacobs University Bremen gGmbH). The author is deeply grateful to Prof. Herbert Jaeger of the Jacobs University Bremen for his precious technical help and for having made this work possible.