采用多头注意力机制的C&amp;RM-MAKT预测算法

王炼红; 罗志辉; 林飞鹏; 李潇瑶

doi:10.12263/DZXB.20220790

您当前的位置：

首页 >

文章列表页 >

采用多头注意力机制的C&RM-MAKT预测算法

学术论文 | 更新时间：2025-12-08

- 采用多头注意力机制的C&RM-MAKT预测算法
- C&RM-MAKT Prediction Algorithm Using Multi-Head Attention Mechanism
- 电子学报 2023年51卷第5期页码：1215-1222
- 作者机构：
  
  湖南大学电气与信息工程学院，湖南长沙 410082
- 作者简介：
  
  [ "王炼红　女，1971年5月生，湖南宁乡人.博士，副教授、硕士生导师.2011年3月至2012年3月，于美国布兰迪斯大学做访问学者.主要研究方向为信号处理、数据挖掘技术、人工智能. E-mail: wanglh@hnu.edu.cn" ]
  [ "罗志辉　男，1998年9月出生于湖南省永州市.2020年于南京农业大学获得工学学士学位.现为湖南大学电气与信息工程学院硕士研究生，主要研究方向为教育数据挖掘和机器学习.E-mail: luo1998@hnu.edu.cn" ]
- 基金信息：
  
  国家重点研发计划(2019YFE0105300);中国高等教育学会数字化课程资源专项研究课题(21SZYB15)
- DOI：10.12263/DZXB.20220790
  中图分类号： TP391
- 收稿：2022-07-08，
  
  修回：2022-10-03，
  
  纸质出版：2023-05-25
- 稿件说明：
移动端阅览
王炼红,罗志辉,林飞鹏等.采用多头注意力机制的C&RM-MAKT预测算法[J].电子学报,2023,51(05):1215-1222.

WANG Lian-hong,LUO Zhi-hui,LIN Fei-peng,et al.C&RM-MAKT Prediction Algorithm Using Multi-Head Attention Mechanism[J].ACTA ELECTRONICA SINICA,2023,51(05):1215-1222.
王炼红,罗志辉,林飞鹏等.采用多头注意力机制的C&RM-MAKT预测算法[J].电子学报,2023,51(05):1215-1222. DOI： 10.12263/DZXB.20220790.

WANG Lian-hong,LUO Zhi-hui,LIN Fei-peng,et al.C&RM-MAKT Prediction Algorithm Using Multi-Head Attention Mechanism[J].ACTA ELECTRONICA SINICA,2023,51(05):1215-1222. DOI： 10.12263/DZXB.20220790.

摘要

针对深度知识追踪模型中普遍存在知识状态向量可解释性弱、缺失历史序列数据语义特征信息、忽视历史序列数据对预测结果影响程度等问题，本文提出了一种融合认知诊断理论和多头注意力机制的预测模型C

RM-MAKT（Cognitive

Response Model- Multi-head Attention Knowledge Tracing）.C

RM-MAKT采用Word2Vec和BiLSTM（Bi-directional Long Short-Term Memory）网络将时序数据变换为低维连续实值向量，引入C

RM训练出的可解释性参数来建模学生学习状态，在模型机理层面将知识状态向量扩展为知识状态矩阵.最后，C

RM-MAKT使用多头注意力机制计算出历史序列数据对预测结果的影响程度，以提高模型的可解释性与精度.预测实验结果表明：C

RM-MAKT在HNU_SYS1、HNU_SYS2、Math1和Frcsub四个数据集上都取得了最佳性能结果，尤其在HNU_SYS2中，C

RM-MAKT相较于现有知识追踪模型在AUC（Area Uder the Curve）、ACC（ACCuracy）和

（

-Measure）指标上分别提升了4.3%、3.6%和5.9%.此外，HNU_SYS2数据集上的可解释性分析表明：C

RM-MAKT模型内部参数可解释性强，一定程度上缓解了深度模型的“黑箱”特性.

Abstract

To address the problems of weak interpretability of knowledge state vectors

lackness of the semantic feature of historical sequence data

and failure to consider the influence of historical sequence data on performance prediction in existing deep knowledge tracking models

this paper proposes a predictive model C

RM-MAKT (Cognitive

Response Model-Multi-head Attention Knowledge Tracing) integrating cognitive diagnostic theory with multiple attention mechanisms. C

RM-MAKT uses Word2Vec and BiLSTM (Bi-directional Long Short-Term Memory) networks to transform the time series data into low-dimensional continuous real vectors

and applies C

RM to pre-train the interpretable parameters for student state modeling

and extends the knowledge state vectors into a knowledge state matrix at the model mechanism level. C

RM-MAKT utilizes multiheaded attention mechanism to estimate the influence degree of historical exercises on the performance prediction to improve the interpretability and accuracy of the model. The prediction experiment results show that C

RM-MAKT performs the best on datasets HNU_SYS1

HNU_SYS2

Math1

and Frcsub. Especially on dataset HNU_SYS2

RM-MAKT improves the existing knowledge tracking models by 4.3%

3.6%

and 5.9% in terms of AUC (Area Uder the Curve)

ACC (ACCuracy)

and

(

-Measure)

respectively. In addition

according to the interpretability analysis on dataset HNU_SYS2

the internal parameters of the C

RM-MAKT model are highly interpretable

which alleviates the “black box” characteristics of the deep model to a certain extent.

关键词

Keywords

references

FAN X . Item response theory and classical test theory: An empirical comparison of their item/person statistics [J]. Educational and Psychological Measurement , 1998 , 58 ( 3 ): 357 - 381 .

DE LA TORRE J . DINA model and parameter estimation: A didactic [J]. Journal of Educational and Behavioral Statistics , 2009 , 34 ( 1 ): 115 - 130 .

HARTZ M C . A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality [J]. American Journal of Gastroenterology , 2002 , 95 ( 4 ): 906 - 909 .

CORBETT A T , ANDERSON J R . Knowledge tracing: Modeling the acquisition of procedural knowledge [J]. User Modeling and User-Adapted Interaction , 1994 , 4 ( 4 ): 253 - 268 .

HAWKINS W J , HEFFERNAN N T . Using similarity to the previous problem to improve Bayesian knowledge tracing [C]// Proceedings of the Workshops held at Educational Data Mining 2014 (WSEDM 2014) . London : CEUR-WS , 2014 : 136 - 140 .

AGARWAL D , BAKER R , MURALEEDHARAN A . Dynamic knowledge tracing through data driven recency weights [C]// The 13th International Conference on Educational Data Mining . Morocco : Open Access , 2020 : 725 - 729 .

PIECH C , SPENCER J , HUANG J , et al . Deep knowledge tracing [J]. Computer Science , 2015 , 3 ( 3 ): 19 - 23 .

YEUNG C K , YEUNG D Y . Addressing two problems in deep knowledge tracing via prediction-consistent regularization [C]// Proceedings of the 5th Annual ACM Conference on Learning at Scale . London : ACM , 2018 : 1 - 10 .

MINN S , YI Y , DESMARAIS M C , et al . Deep knowledge tracing and dynamic student classification for knowledge tracing [C]// 2018 IEEE International Conference on Data Mining . Singapore : IEEE , 2018 : 1182 - 1187 .

ZHANG J , SHI X , KING I , et al . Dynamic key-value memory networks for knowledge tracing [C]// Proceedings of the 26th International Conference on World Wide Web . Perth : ACM , 2017 : 765 - 774 .

SUN X , ZHAO X , LI B , et al . Dynamic key-value memory networks with rich features for knowledge tracing [J]. IEEE Transactions on Cybernetics , 2022 , 52 ( 8 ): 8239 - 8245 .

LIU Q , HUANG Z , YIN Y , et al . EKT: Exercise-aware knowledge tracing for student performance prediction [J]. IEEE Transactions on Knowledge and Data Engineering , 2019 , 33 ( 1 ): 100 - 115 .

TORRE J , DOUGLAS J A . Higher-order latent trait models for cognitive diagnosis [J]. Psychometrika , 2004 , 69 ( 3 ): 333 - 353 .

DE L , SONG H . Simultaneous estimation of overall and domain abilities: a higher-order IRT model approach [J]. Applied Psychological Measurement , 2009 , 33 ( 8 ): 620 - 639 .

LIU Q , WU R Z , CHEN E H , et al . Fuzzy cognitive diagnosis for modelling examinee performance [J]. ACM Transactions on Intelligent Systems and Technology , 2018 , 9 ( 4 ): 1 - 26 .

WANG F , LIU Q , CHEN E , et al . Neural cognitive diagnosis for intelligent education systems [C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence . New York : AAAI , 2020 : 6153 - 6161 .

王炼红 , 刘畅 , 周熊 , 等 . 基于学习者认知反应模型的认知诊断方法 : CN202110122198.0 [P]. 2021-05-07 .

GELFAND A E , HILLS S E , RACINE-POON A . Illustration of Bayesian inference in normal data models using Gibbs sampling [J]. Journal of the American Statistical Association , 1990 , 85 ( 412 ): 972 - 985 .

WEN H , DING G , LIU C , et al . Matrix factorization meets cosine similarity: Addressing sparsity problem in collaborative filtering recommender system [C]// The 16th Asia-Pacific Web Conference . Cham : Springer , 2014 : 306 - 317 .

BAG S , KUMAR S K , TIWARI M K . An efficient recommendation generation using relevant Jaccard similarity [J]. Information Sciences , 2019 , 483 : 53 - 64 .

SUBAKAN C , RAVANELLI M , CORNELL S , et al . Attention is all you need in speech separation [C]// 2021 IEEE International Conference on Acoustics, Speech and Signal Processing . Toronto : IEEE , 2021 : 21 - 25 .

SALAKHUTDINOV R , MNIH A . Probabilistic matrix factorization [C]// Advances in Neural Information Processing Systems 20 (NIPS 2007) . Vancouver : ACM , 2007 : 1257 - 1264 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

面向时序异常检测的可变视距多向扫描方法

基于邻域与超图协作的会话推荐

应试心理状态增强的学生表现预测模型

基于EIMYOLO的高分遥感图像目标检测

采用多头注意力机制的C&RM-MAKT预测算法

C&RM-MAKT Prediction Algorithm Using Multi-Head Attention Mechanism

DOI：10.12263/DZXB.20220790

摘要

Abstract

关键词

Keywords

references