In this paper, we propose a deep learning-based system for music detection and speech detection from broadcast contents. In addition, we propose a convolutional neural network to improve the performance of music detection and speech detection.
First, we analyze the characteristics of speech and music in broadcast contents. In the time domain, music has little change, but speech has large change. Accordingly, in the spectral domain, music shows straight horizontal lines but speech shows straight vertical lines. From this observation, we propose a convolutional neural network considering the characteristics of speech and music.
We propose a convolutional layer with Mel-scale kernel for music detection from broadcast contents. The Mel-scale changes the kernel size and the backpropagation algorithm trains the kernel shape. In order to verify the performance of the proposed model, we implement a comparison model. Our proposed model shows the best performance with F-score 89.9% in Korean broadcast data. In addition, our proposed model shows the best performance with 88.9% F-score in Spanish broadcast data, and 86.5% F-score in British broadcast data.
We propose a multi-scale time-dilated convolutional layer for speech detection from broadcast contents. Our proposed model is designed to learn the temporal changes of feature vectors. We implement a comparison model of convolutional neural network (CNN), recurrent neural network (RNN), convolutional recurrent neural network (CRNN), and ResNet architecture for performance comparison with the proposed model. Our proposed model shows the best performance with 91.7% F-score in Korean broadcast data. In addition, our model shows the best performance with 89.3% and 93.1% F-score in the British and Spanish broadcast data, respectively.
These experimental results show that the proposed convolutional neural networks contribute to the improvement of music detection and speech detection performance.