复调音乐主旋律提取方法综述

doi:10.3969/j.issn.0372-2112.2017.04.032

电子学报 ›› 2017, Vol. 45 ›› Issue (4): 1000-1011.DOI: 10.3969/j.issn.0372-2112.2017.04.032

复调音乐主旋律提取方法综述

张维维^1,2, 陈喆¹, 殷福亮¹, 张俊星²

1. 大连理工大学信息与通信工程学院, 辽宁大连 116024;
2. 大连民族大学信息与通信工程学院, 辽宁大连 116600

收稿日期:2016-01-29 修回日期:2016-07-28 出版日期:2017-04-25
- 作者简介:
- 张维维 女,辽宁大连人,1981年5月生,大连理工大学在读博士生,大连民族大学讲师.主要研究方向为音乐信号处理,音乐信息检索.Email:zhangww@dlnu.edu.cn;陈喆男,黑龙江省泰来人,1975年11月生,大连理工大学副教授.主要研究方向为数字信号处理、语音处理、图像处理、宽带无线通信技术.E-mail:zhechen@dlut.edu.cn
- 基金资助:
- 国家自然科学基金 (No.61172107，No.61172110）; 国家863高技术研究发展计划 (No.2015AA016306）; 辽宁省教育厅科学研究一般项目 (No.L2015135）; 中央高校基本科研业务费 (No.DUT13LAB06，No.DC201502060404）

Review on Melody Extraction from Polyphonic Music

ZHANG Wei-wei^1,2, CHEN Zhe¹, YIN Fu-liang¹, ZHANG Jun-xing²

1. School of Information and Communication Engineering, Dalian University of Technology, Dalian 116024, Liaoning, China;
2. School of Information and Communication Engineering, Dalian Minzu University, Dalian 116024, Liaoning, China

Received:2016-01-29 Revised:2016-07-28 Online:2017-04-25 Published:2017-04-25
- Supported by:
- National Natural Science Foundation of China (No.61172107, No.61172110); National High-tech R&D Program of China (863 Program) (No.2015AA016306); Research Project of Education Department of liaoning Province (General Program) (No.L2015135); Fundamental Research Funds for the Central Universities (No.DUT13LAB06, No.DC201502060404)

摘要/Abstract

摘要：

主旋律提取在音乐检索、乐谱抄录、翻唱识别等领域具有广泛应用.复调音乐频谱结构复杂多样，音高变化范围广，因此复调音乐的主旋律提取较为困难.本文综述了复调音乐主旋律提取研究进展，对复调音乐主旋律提取方法进行分类，阐述了典型方法，介绍了主旋律提取的评价指标，给出了最新MIREX主旋律提取评测结果，说明了主旋律提取面临的主要挑战，并对主旋律提取技术发展方向进行了展望.

关键词: 主旋律提取, 复调音乐, 多声部音乐, 音乐信号处理, 音乐信息检索

Abstract:

Melody extraction is to automatically extract the melody from a given piece of polyphonic music,and has been widely applied in music retrieval,score transcription,cover song identification and so on.Melody extraction from digital polyphonic music is reviewed in this paper.Firstly,the methods used for melody extraction are classified.Then,the representative methods are elaborated in detail,evaluation methodologies are presented,and the most recently MIREX melody extraction competition results are shown.Finally,the existing challenges are summarized,and the future research directions are provided.

Key words: melody extraction, polyphonic music, multi-part music, music signal processing, music information retrieval

中图分类号:

TP391

张维维, 陈喆, 殷福亮, 等. 复调音乐主旋律提取方法综述[J]. 电子学报, 2017, 45(4): 1000-1011.

ZHANG Wei-wei, CHEN Zhe, YIN Fu-liang, et al. Review on Melody Extraction from Polyphonic Music[J]. Acta Electronica Sinica, 2017, 45(4): 1000-1011.

参考文献

[1] Poliner G E,Ellis D P,Ehmann A F,et al.Melody transcription from music audio:approaches and evaluation[J].IEEE Transactions on Audio,Speech,and Language Processing,2007,15(4):1247-1256.
[2] 李重光.基本乐理通用教材[M].北京:高等教育出版社,2004. Li Chong-Guang.Basic Music Theory Textbook[M].Beijing:Higher Education Press,2004(in Chinese)
[3] Salamon J,Gomez E,Ellis D P,et al.Melody extraction from polyphonic music signals:approaches,applications,and challenges[J].IEEE Signal Processing Magazine,2014,31(2):118-134.
[4] Maddage N C,Xu C,Kankanhalli M S,et al.Content-based music structure analysis with applications to music semantics understanding[A].12th Annual ACM International Conference on Multimedia[C].New York,USA,1999.112-119.
[5] Rolland P Y,Raškinis G,Ganascia J G.Musical content-based retrieval:an overview of the Melodiscov approach and system[A].7th ACM International Conference on Multimedia[C].Florida:The Special Interest Group on Multimedia,2004.81-84.
[6] Salamon J,Serra J,Gómez E.Tonal representations for music retrieval:from version identification to query-by-humming[J].International Journal of Multimedia Information Retrieval,2013,2(1):45-58.
[7] Salamon J,Rocha B,Gómez E.Musical genre classification using melody features extracted from polyphonic music signals[A].IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP)[C].Toyoto:IEEE Signal Processing Society,2012.81-84.
[8] Tsai W H,Yu H M,Wang H M.Using the similarity of main melodies to identify cover versions of popular songs for music document retrieval[J].Journal of Information Science and Engineering,2008,24(6):1669-1687.
[9] Foucard R,Durrieu J L,Lagrange M,et al.Multimodal similarity between musical streams for cover version detection[A].IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP)[C].Dallas:IEEE Signal Processing Society,2010.5514-5517.
[10] Wu Y D,Li Y,Liu B L.A new method for approximate melody matching[A].International Conference on Machine Learning and Cybernetics[C].Xi'an:IEEE Systems,Man,and Cybernetics Society,2003.2687-2691.
[11] Chang C W,Jiau H C.Extracting significant repeating figures in music by using quantized melody contour[A].IEEE International Symposium on Computers and Communication[C].Antalya:IEEE Communications Society,2003.1061-1066.
[12] Ozcan G,Isikhan C,Alpkocak A.Melody extraction on MIDI music files[A].7th IEEE International Symposium on Multimedia[C].California:IEEE Computer Society,2005.1-8.
[13] Li J,Yang X,Chen Q.MIDI melody extraction based on improved neural network[A].International Conference on Machine Learning and Cybernetics[C].Baoding:IEEE Systems,Man,and Cybernetics Society,2009.1133-1138.
[14] Shih H H,Narayanan S S,Kuo C C.Automatic main melody extraction from MIDI files with a modified Lempel-Ziv algorithm[A].International Symposium on Intelligent Multimedia,Video and Speech Processing[C].Hongkong:IEEE Tainan Section,2001.9-12.
[15] Akant K,Limaye S.Pitch contour extraction of singing voice in polyphonic recordings of Indian classical music[A].International Conference on Electronic Systems,Signal Processing and Computing Technologies[C].Maharashtra:IEEE Computer Society,2014.123-128.
[16] Cao C,Li M,Liu J,et al.Singing melody extraction in polyphonic music by harmonic tracking[A].8th International Society for Music Information Retrieval Conference[C].Vienna:Music Information Retrieval Society,2007.373-374.
[17] Dong M,Chan P,Cen L,et al.Aligning singing voice with MIDI melody using synthesized audio signal[A].7th International Symposium on Chinese Spoken Language Processing[C].Nantou:International Speech Communication Association,2010.95-98.
[18] Fujihara H,Kitahara T,Goto M,et al.F0 estimation method for singing voice in polyphonic audio signal based on statistical vocal model and Viterbi search[A].IEEE International Conference on Acoustics,Speech and Signal Processing[C].Quebec:IEEE Signal Processing Society,2006.253-256.
[19] Hsu C L,Jang J S R.Singing pitch extraction by voice vibrato/tremolo estimation and instrument partial deletion[A].11th International Society for Music Information Retrieval[C].Florida:Music Information Retrieval Society,2010.525-530.
[20] Li M,Li J,Han J,et al.Singing melody extraction from pop songs using a novel feature and Viterbi search[A].International Conference on Computational Intelligence and Software Engineering[C].Wuhan:Engineering Information Institute,2010.1-4.
[21] Yeh T C,Wu M J,Jang J,et al.A hybrid approach to singing pitch extraction based on trend estimation and hidden Markov models[A].IEEE International Conference on Acoustics,Speech and Signal Processing[C].Kyoto:IEEE Signal Processing Society,2012.457-460.
[22] Rao V,Rao P.Vocal melody extraction in the presence of pitched accompaniment in polyphonic music[J].IEEE Transactions on Audio,Speech,and Language Processing,2010,18(8):2145-2154.
[23] Sutton C,Vincent E,Plumbley M D,et al.Transcription of vocal melodies using voice characteristics and algorithm fusion[J].Music Information Retrieval Evaluation eXchange,2006,7(1):1-4.
[24] Vidwans A,Ganguli K K,Rao P.Classification of Indian classical vocal styles from melodic contours[A].2nd CompMusic Workshop[C].Istanbul:Music Technology Group of the Universitat Pompeu Fabra in Barcelona,2012.139-146.
[25] Shao X,Xu C,Kankanhalli M S.Predominant vocal pitch detection in polyphonic music[A].IEEE International Conference on Multimedia and Expo[C].Ontario:IEEE Circuits and Systems Society,2006.897-900.
[26] Yao G,Zheng Y,Xiao L,et al.Efficient vocal melody extraction from polyphonic music signals[J].Elektronika ir Elektrotechnika,2013,19(6):103-108.
[27] Durrieu J L,Richard G,David B,et al.Source/filter model for unsupervised main melody extraction from polyphonic audio signals[J].IEEE Transactions on Audio,Speech,and Language Processing,2010,18(3):564-575.
[28] Gómez E,Klapuri A,Meudic B.Melody description and extraction in the context of music content processing[J].Journal of New Music Research,2003,32(1):23-40.
[29] Joo S,Park S,Jo S,et al.Melody extraction based on harmonic coded structure[A].12th International Society for Music Information Retrieval Conference (ISMIR)[C].Florida:Music Information Retrieval Society,2011.227-232.
[30] Marolt M.Audio melody extraction based on timbral similarity of melodic fragments[A].International Conference on Computer as a Tool[C].Belgrade:IEEE Vancouver Section,2005.1288-1291.
[31] Salamon J,Gómez E.Melody extraction from polyphonic music signals using pitch contour characteristics[J].IEEE Transactions on Audio,Speech,and Language Processing,2012,20(6):1759-1770.
[32] Thornburg H,Leistikow R J,Berger J.Melody extraction and musical onset detection via probabilistic models of framewise STFT peak data[J].IEEE Transactions on Audio,Speech,and Language Processing,2007,15(4):1257-1272.
[33] Han J,Chen C W.Improving melody extraction using probabilistic latent component analysis[A].IEEE International Conference on Acoustics,Speech and Signal Processing[C].Prague:IEEE Signal Processing Society,2011.33-36.
[34] Goto M.A robust predominant-F0 estimation method for real-time detection of melody and bass lines in CD recordings[A].IEEE International Conference on Acoustics,Speech,and Signal Processing[C].Istanbul:IEEE Signal Processing Society,2000.757-760.
[35] Goto M.A real-time music-scene-description system:Predominant-F0 estimation for detecting melody and bass lines in real-world audio signals[J].Speech Communication,2004,43(4):311-329.
[36] Cancela P.Tracking melody in polyphonic audio[A].9th Music Information Retrieval Evaluation eXchange (MIREX 2008)[C].Illinois:The International Music Information Retrieval Systems Evaluation Laboratory,2008.9(1):1-3.
[37] Joo S,Jo S,Yoo C D.Melody extraction from polyphonic audio signal[A].11th Music Information Retrieval Evaluation eXchange (MIREX)[C].Illinois:The International Music Information Retrieval Systems Evaluation Laboratory,2010.11(1):1-4.
[38] Paiva R P,Mendes T,Cardoso A.Melody detection in polyphonic musical signals:Exploiting perceptual rules,note salience,and melodic smoothness[J].Computer Music Journal,2006,30(4):80-98.
[39] Dressler K.Pitch estimation by the pair-wise evaluation of spectral peaks[A].AES 42nd International Conference[C].Ilmenau:Australasian Evaluation Society,2011.1-10.
[40] Ryynänen M P,Klapuri A P.Automatic transcription of melody,bass line,and chords in polyphonic music[J].Computer Music Journal,2008,32(3):72-86.
[41] Arora V,Behera L.On-line melody extraction from polyphonic audio using harmonic cluster tracking[J].IEEE Transactions on Audio,Speech,and Language Processing,2013,21(3):520-530.
[42] Tachibana H,Ono T,Ono N,et al.Melody line estimation in homophonic music audio signals based on temporal-variability of melodic source[A].IEEE International Conference on Acoustics,Speech and Signal Processing[C].Dallas:IEEE Signal Processing Society,2010.425-428.
[43] Ellis D P,Poliner G E.Classification-based melody transcription[J].Machine Learning,2006,65(2-3):439-456.
[44] Song L,Li M.Bayesian framework-based vocal melody extraction for MIREX2014[A].Music Information Retrieval Evaluation eXchange (MIREX)[C].Illinois:The International Music Information Retrieval Systems Evaluation Laboratory,2014.15(1):1-2.
[45] Jo S,Yoo C D,Doucet A.Melody tracking based on sequential bayesian model[J].IEEE Journal of Selected Topics in Signal Processing,2011,5(6):1216-1227.
[46] Yoon J Y,Song C J,Lee S P,et al.Extracting predominant melody of polyphonic music based on harmonic structure[A].7th Music Information Retrieval Evaluation eXchange (MIREX)[C].Illinois:The International Music Information Retrieval Systems Evaluation Laboratory,2011.7(1):1-2.
[47] Ryynänen M,Virtanen T,Paulus J,et al.Accompaniment separation and Karaoke application based on automatic melody transcription[A].IEEE International Conference on Multimedia and Expo[C].Hannover:IEEE Circuits and Systems Society,2008.1417-1420.
[48] Zhu Y,Gao S.Extracting vocal melody from Karaoke music audio[A].IEEE International Conference on Multimedia and Expo[C].Amsterdam:IEEE Circuits and Systems Society,2005.1-5.
[49] Bosch J J,Gómez E.Melody extraction in symphonic classical music:a comparative study of mutual agreement between humans and algorithms[A].9th Conference on Interdisciplinary Musicology[C].Berlin:World Academy of Science,Engineering and Technology,2014.1-6.
[50] Tang Z,Black D A.Melody extraction from polyphonic audio of western opera:a method based on detection of the singer's formant[A].7th International Society for Music Information Retrieval Conference (ISMIR)[C].Taipei:Music Information Retrieval Society,2014.161-166.
[51] Dressler K.Audio melody extraction for MIREX2014[A].Music Information Retrieval Evaluation eXchange (MIREX)[C].Illinois:The International Music Information Retrieval Systems Evaluation Laboratory,2014.15(1):1-3.
[52] Ikemiya Y,Yoshii K,Itoyama K.MIREX2014:audio melody extraction[A].Music Information Retrieval Evaluation eXchange (MIREX)[C].Illinois:The International Music Information Retrieval Systems Evaluation Laboratory,2014.15(1):1-2.
[53] Degani A,Leonardi R,Migliorati P,et al.A pitch salience function derived from harmonic frequency deviations for polyphonic music analysis[A].17th International Conference on Digital Audio Effects[C].Erlangen:Fraunhofer Institut für Integrierte Schaltungen,2014.1-7.
[54] Chien Y R,Wang H M,Jeng S K.An acoustic-phonetic model of F0 likelihood for vocal melody extraction[J].IEEE/ACM Transactions on Audio,Speech and Language Processing,2015,23(9):1457-1468.
[55] Cancela P,Rocamora M,López E.An efficient multi-resolution spectral transform for music analysis[A].10th International Society for Music Information Retrieval[C].Kobe:Music Information Retrieval Society,2009.309-314.
[56] Park S,Jo S,Yoo C D.Melody extraction from polyphonic audio signal MIREX 2011[A].7th Music Info Retrieval Evaluation eX-change (MIREX)[C].Illinois:The International Music Information Retrieval Systems Evaluation Laboratory,2011.102(1):1-2.
[57] Charpentier F.Pitch detection using the short-term phase spectrum[A].IEEE International Conference on Acoustics,Speech,and Signal Processing (ICASSP)[C].Tokyo:IEEE Signal Processing Society,1986.113-116.
[58] Flanagan J L,Golden R.Phase vocoder[J].Bell System Technical Journal,1966,45(9):1493-1509.
[59] Salamon J,Gómez E,Bonada J.Sinusoid extraction and salience function design for predominant melody estimation[A].14th International Conference on Digital Audio Effects[C].Paris:The National Center for Scientific Research,2011.73-80.
[60] Klapuri A P.Multiple fundamental frequency estimation by summing harmonic amplitudes[A].7th International Society for Music Information Retrieval Conference (ISMIR)[C].Victoria:Music Information Retrieval Society,2006.216-221.
[61] Klapuri A P.Multiple fundamental frequency estimation based on harmonicity and spectral smoothness[J].IEEE Transactions on Speech and Audio Processing,2003,11(6):804-816.
[62] Hermes D J.Measurement of pitch by subharmonic summation[J].Journal of the Acoustical Society of America,1988,83(1):257-264.
[63] Hsu C L,Chen LY,Jang J S R,et al.Singing pitch extraction from monaural polyphonic songs by contextual audio modeling and singing harmonic enhancement[A].10th International Society for Music Information Retrieval Conference[C].Kobe:Music Information Retrieval Society,2009.201-206.
[64] Chien Y R,Wang H M,Jeng S K.Vocal melody extraction based on an acoustic-phonetic model of pitch likelihood[A].Music Information Retrieval Evaluation eXchange (MIREX)[C].Illinois:The International Music Information Retrieval Systems Evaluation Laboratory,2014.15(1):1-2.
[65] Hsu C L,Wang D,Jang J S.A trend estimation algorithm for singing pitch detection in musical recordings[A].IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP)[C].Prague:IEEE Signal Processing Society,2011.393-396
[66] 龚君才,刘刚.一种基于隐马尔科夫模型的波形文件主旋律基频提取算法[J].软件,2013,34(12):152-155. Gong Jun-Cai,Liu Gang.A melody pitch extraction algorithm for waveform file based on hidden Markov mode[J].Software,2013,34(12):152-155.(in Chinese)
[67] Li Y,Wang D.Separation of singing voice from music accompaniment for monaural recordings[J].IEEE Transactions on Audio,Speech,and Language Processing,2007,15(4):1475-1487.
[68] Wang Y,Ou Z.Combining HMM-based melody extraction and NMF-based soft masking for separating voice and accompaniment from monaural audio[A].IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP)[C].Prague:IEEE Signal Processing Society,2011.1-4.
[69] 宋岳阳.基于单源欠定语音分离的音乐主旋律提取方法研究[D].北京:北京邮电大学,2012. Song Yue-Yang.An approach for music melody extraction based on underdetermined single-source speech separation[D].Beijing:Beijing University of Posts and Telecommunications,2012.(in Chinese)
[70] Poliner G E,Ellis D P.A classification approach to melody transcription[A].6th International Conference on Music Information Retrieval (ISMIR)[C].London:Music Information Retrieval Society,2005.161-166.
[71] Ryynänen M,Klapuri A.Transcription of the singing melody in polyphonic music[A].International Society for Music Information Retrieval (ISMIR)[C].Victoria:Music Information Retrieval Society,2006.222-227.
[72] Fan Z,Jang J,Lu C.Singing voice separation and pitch extraction from monaural polyphonic audio music via DNN and ddaptive pitch tracking[A].2nd IEEE International Conference on Multimedia Big Data[C].Taipei:Academia Sinica,2016.1-8.
[73] http://www.music-ir.org/mirex/wiki/MIREX_-HOME.
[74] Chien Y R,Wang H M,Jeng S K.Simulated formant modeling of accompanied singing signals for vocal melody extraction[A].9th Sound and Music Computing Conference (SMC)[C].Copenhagen:Logos Verlag Berlin GmbH,2012.33-40.
[75] Huang P S,Chen S D,P Smaragdis,et al.Singing voice separation from monaural recordings using robust principal component analysis[A].IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP)[C].Toyoto:IEEE Signal Processing Society,2012.57-60.
[76] Ikemiya Y,Yoshii K,Itoyama K.Singing voice analysis and editing based on mutually dependent F0 estimation and source separation[A].IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP)[C].Brisbane:IEEE Signal Processing Society,2015.574-578.
[77] Bosch J,Gómez E.Melody extraction by means of a source-filter model and pitch contour characterization[A].Music Informormation Retrieval Evaluation eXchange (MIREX)[C].Illinois:The International Music Information Retrieval Systems Evaluation Laboratory,2015.16(1):1-3.

复调音乐主旋律提取方法综述

Review on Melody Extraction from Polyphonic Music

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics