Panel or jury testing methods in a metrological perspective

Panel or jury testing is a widely used technique in the measurement of quantities related to human perception. However, despite both scientific and applied interests, the confidence of potential users of this technique is less than in more familiar physical and engineering measurements. This lack of confidence arises mainly due to the lack of standardization and of internationally recognized metrological references for panel testing. Here we discuss some methodological aspects of panel testing, including the construction of the reference scale, the definition and realization of a measuring system based on a previously established scale, its calibration and related uncertainty evaluation. We provide illustrative examples concerning the measurement of the quality of sound (acceptability) of machinery used for the automatic sorting of post, presenting experimental results from tests performed in our laboratory, which seem encouraging. Finally, we propose some possible actions for standardization and for the promotion of a metrological approach.