Binaural model for synthetic sound image direction perception using a convolutional neural network (Verification on simple simulation of in-car sound field)

A binaural model for sound image direction perception with a single-source and synthetic sound image direction perception with two-source reproduction in a simple simulation of an in-car sound field using deep learning was verified. As input data for training, binaural input signals were simulated from sound sources (speech) surrounding the listener, considering the simple simulation sound field, including the interior of the car and the head-related transfer function. The image data were obtained by time-frequency transformation into a log-Mel spectrogram. The sound image direction corresponding to each input data point was considered as the correct data, and a convolutional neural network was used to train a model for estimating the sound image direction. As a result of deep learning, the model performance was not significantly affected by the sound image direction perception of a single or two sound sources, and it was confirmed that for high-precision classification, it is important that the direct sound be greater than the maximum value of the reflected sound, but the reverberation time is not. In addition, the proposed model was compared with conventional models for in-car sound fields constructed by the author (statistical model and a shallow neural network model), and the proposed model was analyzed using explainable artificial intelligence and findings from previous studies.