A New Model Selection Criterion Based on Kullback-Leibler Information Divergence and Geometric Curvature

YANG Jian; LUO Si-wei; LIU Yun-hui

您当前的位置：

首页 >

文章列表页 >

A New Model Selection Criterion Based on Kullback-Leibler Information Divergence and Geometric Curvature

更新时间：2025-07-16

- A New Model Selection Criterion Based on Kullback-Leibler Information Divergence and Geometric Curvature
- Acta Electronica Sinica Vol. 33, Issue 12, Pages: 2272-2077(2005)
- 作者机构：
  
  北京交通大学计算机与信息技术学院计算机研究所,北京,100044
- 作者简介：
- 基金信息：
- DOI：
  CLC： TP183
- Published：2005
- 稿件说明：
移动端阅览
YANG Jian, LUO Si-wei, LIU Yun-hui. A New Model Selection Criterion Based on Kullback-Leibler Information Divergence and Geometric Curvature[J]. Acta Electronica Sinica, 2005, 33(12): 2272-2077.
DOI：

YANG Jian, LUO Si-wei, LIU Yun-hui. A New Model Selection Criterion Based on Kullback-Leibler Information Divergence and Geometric Curvature[J]. Acta Electronica Sinica, 2005, 33(12): 2272-2077. DOI：

摘要

模型选择的目标就是识别产生给定数据的模型.通常模型的好坏由模型的泛化能力来度量

而泛化能力包含模型对给定数据的拟合度和模型复杂度两个方面.本文从信息几何的观点使用定义在流形上的广义KL距离来度量模型的拟合度;另一方面从微分几何的观点用曲率的概念来度量模型的内在复杂度;因此

拟合度和复杂度的表示都具有在参数变换下保持不变的特点.通过理论分析

我们证明了用于表示模型预测能力的未来残差与模型固有曲率的关系.由此提出了一种新的基于广义KL距离和曲率的模型选择准则KLCIC.该准则不仅考虑了样本大小、参数个数和函数形式等影响复杂度的因素

而且具有非常清晰直观的几何意义.实验结果表明该方法的有效性.

Abstract

The goal of model selection is to identify the model that generated the data.Goodness of a model is measured using generalizability

which takes two opposite aspects:goodness of fit and model complexity into account.we use generalized KL-divergence defined on the manifold to measure the goodness of fit

and use the conception of curvature from the point of view of differential geometry to explore the intrinsic model complexity that is free of reparametrization;through theoretical analysis

we proved the future residual that is qualified to measure the generalizability can be expressed by using the intrinsic curvature array of model

from which we propose a new model selection criterion KL-divergence and curvature information criterion with very clear and intuitive geometric understanding of model selection.Experimental results reveal its validity.

关键词

Keywords

references

Views

901

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Variational Learning for Finite Inverted Dirichlet Mixture Models and Applications

Variational Learning for Finite Beta-Liouville Mixture Models and Its Application

An Unsupervised Image Segmentation Algorithm Based on HMRF Model

Variational Bayesian Learning for Beta Mixture Model and Its Application

Related Author

YANG Yi-xian

GUO Chun

GUO Yu-cui

DING Hong-wei

ZHOU Ya-jian

LAI Yu-ping

SHENG Bo-chang

WANG Chang-hong

Related Institution

School of Science Beijing University of Posts and Telecommunications Beijing China

School of Information Science and Engineering Yunnan University Kunming Yunnan China

Information Security Center Beijing University of Posts and Telecommunications Beijing China

School of Science, Beijing University of Posts and Telecommunications

School of Information Science and Engineering, Yunnan University

⁰