Supervised learning relies on the availability and reliability of the labels used to train computational models. In research areas such as Afective Computing and Social Signal Processing, such labels are usually extracted from multiple self-and/or external assessments. Labels are, then, either aggregated to produce a single groundtruth label, or all used during training, potentially resulting in degrading performance of the models. Defning a “true” label is, however, complex. Labels can be gathered at diferent times, with diferent tools, and may contain biases. Furthermore, multiple assessments are usually available for a same sample with potential contradictions. Thus, it is crucial to devise strategies that can take advantage of both self-and external assessments to train computational models without a reliable groundtruth. In this study, we designed and tested 3 of such strategies with the aim of mitigating the biases and making the models more robust to uncertain labels. Results show that the strategy based on weighting the loss during training according to a measure of disagreement improved the performances of the baseline, hence, underlining the potential of such an approach.

Training Computational Models of Group Processes without Groundtruth: the Self- vs External Assessment’s Dilemma

Gualtiero Volpe;Giovanna Varni
2022-01-01

Abstract

Supervised learning relies on the availability and reliability of the labels used to train computational models. In research areas such as Afective Computing and Social Signal Processing, such labels are usually extracted from multiple self-and/or external assessments. Labels are, then, either aggregated to produce a single groundtruth label, or all used during training, potentially resulting in degrading performance of the models. Defning a “true” label is, however, complex. Labels can be gathered at diferent times, with diferent tools, and may contain biases. Furthermore, multiple assessments are usually available for a same sample with potential contradictions. Thus, it is crucial to devise strategies that can take advantage of both self-and external assessments to train computational models without a reliable groundtruth. In this study, we designed and tested 3 of such strategies with the aim of mitigating the biases and making the models more robust to uncertain labels. Results show that the strategy based on weighting the loss during training according to a measure of disagreement improved the performances of the baseline, hence, underlining the potential of such an approach.
File in questo prodotto:
File Dimensione Formato  
2022_conferences_icmi_1.pdf

accesso chiuso

Descrizione: Contributo in atti di convegno
Tipologia: Documento in versione editoriale
Dimensione 760.82 kB
Formato Adobe PDF
760.82 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11567/1103359
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact