有限贝塔刘维尔混合模型的变分学习及其应用

doi:10.3969/j.issn.0372-2112.2014.07.015

电子学报 ›› 2014, Vol. 42 ›› Issue (7): 1347-1352.DOI: 10.3969/j.issn.0372-2112.2014.07.015

有限贝塔刘维尔混合模型的变分学习及其应用

赖裕平¹, 丁洪伟², 周亚建¹, 郭玉翠³, 杨义先¹

1. 北京邮电大学计算机学院, 北京 100876;
2. 云南大学信息学院, 云南昆明 650091;
3. 北京邮电大学理学院, 北京 100876

收稿日期:2013-06-28 修回日期:2014-03-31 出版日期:2014-07-25
- 作者简介:
- 赖裕平 男，1983年10月生于江西瑞金，北京邮电大学博士研究生，主要研究方向为模式识别、机器学习、非高斯统计模型、贝叶斯网络、图像分类.E-mail：Laiyp2009@126.com；丁洪伟 男，1964年5月生于云南景洪，云南大学信息学院副教授，博士研究生，主要研究方向为轮询系统、随机多址通信系统、优化算法研究.；周亚建 男，1971年生于陕西镇安.北京邮电大学副教授、硕士生导师，主要研究方向为移动通信、无线传感网络、信息安全等.；郭玉翠 女，1962年5月生，北京邮电大学理学院教授.研究方向为微分方程边值问题的分析与求解.；杨义先 男，1961年3月生，北京邮电大学计算机学院教授、博士生导师，长江学者，主要研究方向为现代密码基础理论，信息隐慝技术与理论.
- 基金资助:
- 国家自然科学基金 (No.61121061，No.60972077，No.61072079，No.61363085）; 教育部博士点基金 (No.20120005110017）; 国家863高技术研究发展计划 (No.2009AA01Z430）

Variational Learning for Finite Beta-Liouville Mixture Models and Its Application

LAI Yu-ping¹, DING Hong-wei², ZHOU Ya-jian¹, GUO Yu-cui³, YANG Yi-xian¹

1. Information Security Center, Beijing University of Posts and Telecommunications, Beijing 100876, China;
2. School of Information Science and Engineering, Yunnan University, Kunming, Yunnan 650091, China;
3. School of Science, Beijing University of Posts and Telecommunications, Beijing 100876, China

Received:2013-06-28 Revised:2014-03-31 Online:2014-07-25 Published:2014-07-25
- Supported by:
- National Natural Science Foundation of China (No.61121061, No.60972077, No.61072079, No.61363085); Ph.D. Programs Foundation of Ministry of Education of China (No.20120005110017); National High-tech R&D Program of China (863 Program) (No.2009AA01Z430)

摘要/Abstract

摘要：

由于贝塔刘维尔分布的共轭先验分布中存在积分表达式，贝叶斯估计有限贝塔刘维尔混合模型参数异常困难.本文提出利用变分贝叶斯学习模型参数，采用gamma分布作为近似的先验分布并使用合理的非线性近似技术，得到了后验分布的近似解.与常用的EM算法相比，该方法能够同时估计模型参数和确定分量数，且避免了过拟合的问题.在合成数据集及场景分类问题上进行了大量的实验，实验结果验证了本文所提方法的有效性.

关键词: 有限混合模型, 贝塔刘维尔分布, 分解近似, 模型选择, 变分推断

Abstract:

Since the integration expression is present in the conjugate prior distribution,Bayesian estimation of the parameters in finite Beta-Liouville mixture models(BLM)is analytically intractable.In this paper,an approach based on the variational inference framework is proposed.Adopting gamma distributions to approximate the prior distributions of the parameter in BLM and using some reasonable non-linear approximations;the closed form solution for the posterior distribution of the parameters is obtained.Compared to the conventional expectation maximization(EM)algorithm,the proposed algorithm is able to simultaneously estimate the model parameters and determine the number of components;our method also avoids the problem of overfitting.Extensive experimental results based on the synthetic data sets and scenes classification show that the proposed method is efficient and feasible in terms of parameter estimation and model selection.

Key words: finite mixture models, Beta-Liouville distribution, factorized approximation, model selection, variational inference

中图分类号:

TP391.4

赖裕平, 丁洪伟, 周亚建, 等. 有限贝塔刘维尔混合模型的变分学习及其应用[J]. 电子学报, 2014, 42(7): 1347-1352.

LAI Yu-ping, DING Hong-wei, ZHOU Ya-jian, et al. Variational Learning for Finite Beta-Liouville Mixture Models and Its Application[J]. Acta Electronica Sinica, 2014, 42(7): 1347-1352.

参考文献

[1] Everitt B S,Hand D J.Finite Mixture Distributions [M].London:Chapman & Hall,1981.
[2] McLachlanG,Peel D.Finite Mixture Models [M].New York:John Wiley,2000.
[3] Taoufik Bdiri,Nizar Bouguila.Positive vectors clus-ting using interted Dirichlet finite mixture Models [J].Expert Systems with Applications,2012,39(2):1869-1882.
[4] Nizar Bouguila.Hybrid generative/discriminative approaches for proportional data modeling and classification [J].IEEE Transactions on Knowledge and Data Engineering,2012,24(12):2184-2202.
[5] Bishop C M.Pattern Recognition and Machine Learning [M].New York:Springer-Verlag,2006.
[6] Akike H.A new look at the statistical model iden-tification [J].IEEE Transactions on Automatic Control,1974,19(6):716-723.
[7] Schwarz G.Estimating the dimension of a model [J].The Annals of Statistics,1976,6(2):461-464.
[8] 刘伟峰,杨爱兰.基于BIC准则和Gibbs采样的有限混合模型无监督学习算法[J].电子学报,2011,39(3A):134-139. LIU Wei-feng,YANG Ai-lan.Unsupervised learning for finite mixture models based on BIC Criterion and Gibbs Sampling[J].Acta Electronica Sinica,2011,39(3A):134-1139.(in Chinese)
[9] Rissanen J.Modeling by shortest data description[J].Automatica,1978,14(5):456-471.
[10] Ma Zhan-yu,Arne Leijon.Bayesian estimation of beta mixture models with variational inference [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(33):2160-2173.
[11] Corduneanu A,Bishop C.Variational bayesian model selection for mixture distributions [A].In Proceeding of the 8th International Conference on Artificial Intell-igece and Statistics[C].USA:IEEE,2001.27-34.
[12] FanWen-tao,Nizar Bouguila,Djemel Ziou.Variaion-al learning for finite Dirichlet mixture models and applications[J].IEEE Transaction on Neural Net-works and Learning Systems,2012,23(5):762-774.
[13] JosefSivic,Andrew Zisserman.Video google:A text retrieval approach to object matching in videos[A].Proceedings of Ninth IEEE International Conference on Computer Vision[C].2003.USA:IEEE,2003.1470-177.
[14] Jiang Yu-Gang,Ngo Chong-wah,Yang Jun.Towards optimal bag-of-features for object categorization and semantic video retrieval [A].Proceedings of the 6th ACM International Conference on Image and Video Retrieval[C].USA:ACM,2007.494-501.
[15] Aude Oliva,Antonio Torralba.Modeling the shape of the scene:a holistic representation of the spatial envelope [J].International Journal of Computer Vision,2011,42(3):145-175.
[16] David M Blei,Andrew Y Ng,Michael I Jordan.Latent dirichlet allocation [J].Journal of Machine Learning Research,2003,3(33):993-1022.

有限贝塔刘维尔混合模型的变分学习及其应用

Variational Learning for Finite Beta-Liouville Mixture Models and Its Application

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 9

编辑推荐

Metrics

[1]	张浩, 胡昌华, 杜党波, 裴洪, 张建勋. 多状态影响下基于Bi‑LSTM网络的锂电池剩余寿命预测方法[J]. 电子学报, 2022, 50(3): 619-624.
[2]	赖裕平, 高宁, 何闻达, 平原, 杜春来, 王宝成, 丁洪伟. 贝塔混合模型的变分贝叶斯学习及应用[J]. 电子学报, 2018, 46(7): 1787-1792.
[3]	王旭, 何凡, 冯辉, 杨涛, 胡波. MU-MIMO-OFDM上行频偏与信道联合跟踪[J]. 电子学报, 2014, 42(7): 1256-1261.
[4]	赖裕平, 周亚建, 丁洪伟, 郭玉翠, 郭春, 杨义先. 混合逆狄利克雷分布的变分学习及应用[J]. 电子学报, 2014, 42(7): 1435-1440.
[5]	张润生, 李艳斌, 李啸天. 基于合并分层聚类的网络拓扑推断算法[J]. 电子学报, 2013, 41(12): 2346-2352.
[6]	刘伟峰;杨爱兰. 基于BIC准则和Gibbs采样的有限混合模型无监督学习算法[J]. 电子学报, 2011, 39(3A): 134-139.
[7]	常群, 王晓龙, 林沂蒙, 王熙照, Daniel S.Yeung, . 支持向量分类和多宽度高斯核[J]. 电子学报, 2007, 35(3): 484-487.
[8]	朱国普;曾庆双;屈彦呈;王常虹;沈博昌. 一种基于HMRF模型的无监督图像分割算法[J]. 电子学报, 2006, 34(2): 374-379.
[9]	杨坚;罗四维;刘蕴辉. 一种基于广义KL距离和几何曲率的模型选择准则[J]. 电子学报, 2005, 33(12): 2272-2077.