Assessment of human and machine performance in acoustic scene classification: DCASE 2016 case study

Mesaros, Annamaria; Heittola, Toni; Virtanen, Tuomas
Abstract

Human and machine performance in acoustic scene classification is examined through a parallel experiment using TUT Acoustic Scenes 2016 dataset. The machine learning perspective is presented based on the systems submitted for the 2016 challenge on Detection and Classification of Acoustic Scenes and Events. The human performance, assessed through a listening experiment, was found to be significantly lower than machine performance. Test subjects exhibited different behavior throughout the experiment, leading to significant differences in performance between groups of subjects. An expert listener trained for the task obtained similar accuracy to the average of submitted systems, comparable also to previous studies of human abilities in recognizing everyday acoustic scenes.

Keywords

acoustic scene classification; machine learning; human performance; listening experiment

Research areas

Year:
2017
Book title:
2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)
Pages:
319–323
Address:
United States
ISBN:
978-1-5386-1631-4
DOI:
10.1109/WASPAA.2017.8170047