HRTF Individualization From Photogrammetry
HRTFs (head-related transfer functions) are crucial for synthesizing high-fidelity 3D sound over headphones and depends on the individualized relationship between human outer ears and their acoustic properties. The physical characteristics of each outer ear is as unique as a fingerprint and these along with the torso and head all influence external sounds on their way to the ear drum and they do so in a directionally dependent manner. It is this acoustic filtering, mathematically described by head-related impulse response (HRIR) filters, that provides the auditory cues which enable human spatial hearing. Each listener requires his/her own HRIR filters to obtain high-fidelity 3D audio via earphones. HRTFs are the frequency domain representation of HRIRs.
Photogrammetry refers to the three-dimensional reconstruction of a surface mesh of an object from photographs taken from multiple viewpoints. CARLab's photogrammetry rig is composed of 44 synchronized DSLR cameras arranged in a hemisphere that can capture an instantaneous image set of a person. This, combined with close-up mobile phone videos of both ears can produce a detailed 3D morphological surface mesh. FM-BEM numerical simulation executed on this mesh can generate the person's individualized head-related transfer function (HRTF).
The gold standard for producing 3D surface meshes for many years has been from Magnetic Resonance Imaging (MRI). However, this is time-consuming and inconvenient for a person and not ideal for en masse HRTF generation. CARLab is exploring photogrammetry as an alternative method to acquire individualized HRTFs. Fig. 3 shows comparison between acoustically-measured HRTFs, photogrammetry-generated HRTFs and MRI-generated HRTFs.
Photogrammetry refers to the three-dimensional reconstruction of a surface mesh of an object from photographs taken from multiple viewpoints. CARLab's photogrammetry rig is composed of 44 synchronized DSLR cameras arranged in a hemisphere that can capture an instantaneous image set of a person. This, combined with close-up mobile phone videos of both ears can produce a detailed 3D morphological surface mesh. FM-BEM numerical simulation executed on this mesh can generate the person's individualized head-related transfer function (HRTF).
The gold standard for producing 3D surface meshes for many years has been from Magnetic Resonance Imaging (MRI). However, this is time-consuming and inconvenient for a person and not ideal for en masse HRTF generation. CARLab is exploring photogrammetry as an alternative method to acquire individualized HRTFs. Fig. 3 shows comparison between acoustically-measured HRTFs, photogrammetry-generated HRTFs and MRI-generated HRTFs.
Figure 2: The photogrammetry rig
Figure 3: Vertex error (mm) in the photogrammetry ear meshes compared to the MRI ear meshes. A constant luminance colormap has been used for best results.
Figure 4: Comparison between acoustically-measured, photogrammetry-generated and MRI-generated HRTFs. The magnitude of the HRTFs is shown as a function of space for four frequencies.
