A software solution for an automated warping procedure of facial InfraRed Imaging (IRI) on a reference template is presented. Visible and IR videos of six subjects’ faces were simultaneously recorded during controlled movement of the head (i.e., yaw, pitch, frown, speaking). Sixty-eight facial landmarks were identified in each frame in the visible domain through the open source software OpenFace and transformed into the corresponding frame of IRI; the IRI landmarks were then used as basis for a warping procedure into a reference template relying on a Local Weighted Mean (LWM) transformation. The developed algorithm was accurate in both spatial (average Root Mean Square Error, RMSE = 0.66 pixel) and temporal (RMSE = 0.81 ms) co-registration ensuring reliable estimation of regional temperature (RMSE = 0.09 °C) within a defined range of head rotation (±24.23° for yaw and ±13.79° for pitch movements). The proposed algorithm represents a novel automated solution for warping of IRI video sequences of the face on a reference template, pursuing a strong automation of the processing stream, operator-independence, and objectivity in the extraction and analysis of functional thermal data. It permits to overcome current limitations of IRI data processing, mainly due to operator-dependent identification of the Regions of Interest (ROIs). The methodology may allow further development of contact-less evaluation of the psychophysiological state of the subject through IRI being robust to inter-subject geometrical differences of the face and intra-subject movement-related noise.

Automated warping procedure for facial thermal imaging based on features identification in the visible domain

Cardone D.
Primo
;
Perpetuini D.;Filippini C.;Chiarelli A. M.
Penultimo
;
Merla A.
Ultimo
2021-01-01

Abstract

A software solution for an automated warping procedure of facial InfraRed Imaging (IRI) on a reference template is presented. Visible and IR videos of six subjects’ faces were simultaneously recorded during controlled movement of the head (i.e., yaw, pitch, frown, speaking). Sixty-eight facial landmarks were identified in each frame in the visible domain through the open source software OpenFace and transformed into the corresponding frame of IRI; the IRI landmarks were then used as basis for a warping procedure into a reference template relying on a Local Weighted Mean (LWM) transformation. The developed algorithm was accurate in both spatial (average Root Mean Square Error, RMSE = 0.66 pixel) and temporal (RMSE = 0.81 ms) co-registration ensuring reliable estimation of regional temperature (RMSE = 0.09 °C) within a defined range of head rotation (±24.23° for yaw and ±13.79° for pitch movements). The proposed algorithm represents a novel automated solution for warping of IRI video sequences of the face on a reference template, pursuing a strong automation of the processing stream, operator-independence, and objectivity in the extraction and analysis of functional thermal data. It permits to overcome current limitations of IRI data processing, mainly due to operator-dependent identification of the Regions of Interest (ROIs). The methodology may allow further development of contact-less evaluation of the psychophysiological state of the subject through IRI being robust to inter-subject geometrical differences of the face and intra-subject movement-related noise.
File in questo prodotto:
File Dimensione Formato  
2021_Cardone_INPHY.pdf

Solo gestori archivio

Descrizione: Article
Tipologia: PDF editoriale
Dimensione 8.87 MB
Formato Adobe PDF
8.87 MB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11564/741085
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 15
  • ???jsp.display-item.citation.isi??? 13
social impact