C&amp;RM-MAKT Prediction Algorithm Using Multi-Head Attention Mechanism

WANG Lian-hong; LUO Zhi-hui; LIN Fei-peng; LI Xiao-yao

doi:10.12263/DZXB.20220790

您当前的位置：

首页 >

文章列表页 >

C&RM-MAKT Prediction Algorithm Using Multi-Head Attention Mechanism

PAPERS | 更新时间：2025-12-08

- C&RM-MAKT Prediction Algorithm Using Multi-Head Attention Mechanism
- ACTA ELECTRONICA SINICA Vol. 51, Issue 5, Pages: 1215-1222(2023)
- 作者机构：
  
  湖南大学电气与信息工程学院，湖南长沙 410082
- 作者简介：
- 基金信息：
  
  Special Research Project on Digital Curriculum Resources of China Association of Higher Education(21SZYB15)
- DOI：10.12263/DZXB.20220790
  CLC： TP391
- Received：08 July 2022，
  
  Revised：2022-10-03，
  
  Published：25 May 2023
- 稿件说明：
移动端阅览
王炼红,罗志辉,林飞鹏等.采用多头注意力机制的C&RM-MAKT预测算法[J].电子学报,2023,51(05):1215-1222.

WANG Lian-hong,LUO Zhi-hui,LIN Fei-peng,et al.C&RM-MAKT Prediction Algorithm Using Multi-Head Attention Mechanism[J].ACTA ELECTRONICA SINICA,2023,51(05):1215-1222.
王炼红,罗志辉,林飞鹏等.采用多头注意力机制的C&RM-MAKT预测算法[J].电子学报,2023,51(05):1215-1222. DOI： 10.12263/DZXB.20220790.

WANG Lian-hong,LUO Zhi-hui,LIN Fei-peng,et al.C&RM-MAKT Prediction Algorithm Using Multi-Head Attention Mechanism[J].ACTA ELECTRONICA SINICA,2023,51(05):1215-1222. DOI： 10.12263/DZXB.20220790.

摘要

针对深度知识追踪模型中普遍存在知识状态向量可解释性弱、缺失历史序列数据语义特征信息、忽视历史序列数据对预测结果影响程度等问题，本文提出了一种融合认知诊断理论和多头注意力机制的预测模型C

RM-MAKT（Cognitive

Response Model- Multi-head Attention Knowledge Tracing）.C

RM-MAKT采用Word2Vec和BiLSTM（Bi-directional Long Short-Term Memory）网络将时序数据变换为低维连续实值向量，引入C

RM训练出的可解释性参数来建模学生学习状态，在模型机理层面将知识状态向量扩展为知识状态矩阵.最后，C

RM-MAKT使用多头注意力机制计算出历史序列数据对预测结果的影响程度，以提高模型的可解释性与精度.预测实验结果表明：C

RM-MAKT在HNU_SYS1、HNU_SYS2、Math1和Frcsub四个数据集上都取得了最佳性能结果，尤其在HNU_SYS2中，C

RM-MAKT相较于现有知识追踪模型在AUC（Area Uder the Curve）、ACC（ACCuracy）和

（

-Measure）指标上分别提升了4.3%、3.6%和5.9%.此外，HNU_SYS2数据集上的可解释性分析表明：C

RM-MAKT模型内部参数可解释性强，一定程度上缓解了深度模型的“黑箱”特性.

Abstract

To address the problems of weak interpretability of knowledge state vectors

lackness of the semantic feature of historical sequence data

and failure to consider the influence of historical sequence data on performance prediction in existing deep knowledge tracking models

this paper proposes a predictive model C

RM-MAKT (Cognitive

Response Model-Multi-head Attention Knowledge Tracing) integrating cognitive diagnostic theory with multiple attention mechanisms. C

RM-MAKT uses Word2Vec and BiLSTM (Bi-directional Long Short-Term Memory) networks to transform the time series data into low-dimensional continuous real vectors

and applies C

RM to pre-train the interpretable parameters for student state modeling

and extends the knowledge state vectors into a knowledge state matrix at the model mechanism level. C

RM-MAKT utilizes multiheaded attention mechanism to estimate the influence degree of historical exercises on the performance prediction to improve the interpretability and accuracy of the model. The prediction experiment results show that C

RM-MAKT performs the best on datasets HNU_SYS1

HNU_SYS2

Math1

and Frcsub. Especially on dataset HNU_SYS2

RM-MAKT improves the existing knowledge tracking models by 4.3%

3.6%

and 5.9% in terms of AUC (Area Uder the Curve)

ACC (ACCuracy)

and

(

-Measure)

respectively. In addition

according to the interpretability analysis on dataset HNU_SYS2

the internal parameters of the C

RM-MAKT model are highly interpretable

which alleviates the “black box” characteristics of the deep model to a certain extent.

关键词

Keywords

references

FAN X . Item response theory and classical test theory: An empirical comparison of their item/person statistics [J]. Educational and Psychological Measurement , 1998 , 58 ( 3 ): 357 - 381 .

DE LA TORRE J . DINA model and parameter estimation: A didactic [J]. Journal of Educational and Behavioral Statistics , 2009 , 34 ( 1 ): 115 - 130 .

HARTZ M C . A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality [J]. American Journal of Gastroenterology , 2002 , 95 ( 4 ): 906 - 909 .

CORBETT A T , ANDERSON J R . Knowledge tracing: Modeling the acquisition of procedural knowledge [J]. User Modeling and User-Adapted Interaction , 1994 , 4 ( 4 ): 253 - 268 .

HAWKINS W J , HEFFERNAN N T . Using similarity to the previous problem to improve Bayesian knowledge tracing [C]// Proceedings of the Workshops held at Educational Data Mining 2014 (WSEDM 2014) . London : CEUR-WS , 2014 : 136 - 140 .

AGARWAL D , BAKER R , MURALEEDHARAN A . Dynamic knowledge tracing through data driven recency weights [C]// The 13th International Conference on Educational Data Mining . Morocco : Open Access , 2020 : 725 - 729 .

PIECH C , SPENCER J , HUANG J , et al . Deep knowledge tracing [J]. Computer Science , 2015 , 3 ( 3 ): 19 - 23 .

YEUNG C K , YEUNG D Y . Addressing two problems in deep knowledge tracing via prediction-consistent regularization [C]// Proceedings of the 5th Annual ACM Conference on Learning at Scale . London : ACM , 2018 : 1 - 10 .

MINN S , YI Y , DESMARAIS M C , et al . Deep knowledge tracing and dynamic student classification for knowledge tracing [C]// 2018 IEEE International Conference on Data Mining . Singapore : IEEE , 2018 : 1182 - 1187 .

ZHANG J , SHI X , KING I , et al . Dynamic key-value memory networks for knowledge tracing [C]// Proceedings of the 26th International Conference on World Wide Web . Perth : ACM , 2017 : 765 - 774 .

SUN X , ZHAO X , LI B , et al . Dynamic key-value memory networks with rich features for knowledge tracing [J]. IEEE Transactions on Cybernetics , 2022 , 52 ( 8 ): 8239 - 8245 .

LIU Q , HUANG Z , YIN Y , et al . EKT: Exercise-aware knowledge tracing for student performance prediction [J]. IEEE Transactions on Knowledge and Data Engineering , 2019 , 33 ( 1 ): 100 - 115 .

TORRE J , DOUGLAS J A . Higher-order latent trait models for cognitive diagnosis [J]. Psychometrika , 2004 , 69 ( 3 ): 333 - 353 .

DE L , SONG H . Simultaneous estimation of overall and domain abilities: a higher-order IRT model approach [J]. Applied Psychological Measurement , 2009 , 33 ( 8 ): 620 - 639 .

LIU Q , WU R Z , CHEN E H , et al . Fuzzy cognitive diagnosis for modelling examinee performance [J]. ACM Transactions on Intelligent Systems and Technology , 2018 , 9 ( 4 ): 1 - 26 .

WANG F , LIU Q , CHEN E , et al . Neural cognitive diagnosis for intelligent education systems [C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence . New York : AAAI , 2020 : 6153 - 6161 .

王炼红 , 刘畅 , 周熊 , 等 . 基于学习者认知反应模型的认知诊断方法 : CN202110122198.0 [P]. 2021-05-07 .

GELFAND A E , HILLS S E , RACINE-POON A . Illustration of Bayesian inference in normal data models using Gibbs sampling [J]. Journal of the American Statistical Association , 1990 , 85 ( 412 ): 972 - 985 .

WEN H , DING G , LIU C , et al . Matrix factorization meets cosine similarity: Addressing sparsity problem in collaborative filtering recommender system [C]// The 16th Asia-Pacific Web Conference . Cham : Springer , 2014 : 306 - 317 .

BAG S , KUMAR S K , TIWARI M K . An efficient recommendation generation using relevant Jaccard similarity [J]. Information Sciences , 2019 , 483 : 53 - 64 .

SUBAKAN C , RAVANELLI M , CORNELL S , et al . Attention is all you need in speech separation [C]// 2021 IEEE International Conference on Acoustics, Speech and Signal Processing . Toronto : IEEE , 2021 : 21 - 25 .

SALAKHUTDINOV R , MNIH A . Probabilistic matrix factorization [C]// Advances in Neural Information Processing Systems 20 (NIPS 2007) . Vancouver : ACM , 2007 : 1257 - 1264 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Variable Horizon Multi-Directional Scanning Method for Time Series Anomaly Detection

Neighborhood and Hypergraph Collaboration for Session-Based Recommendation

Test-taking Psychological State Enhanced Student Performance Prediction Model

Object Detection Based on EIMYOLO for High-Resolution Remote Sensing Images

Related Author

HUANG Yu-zhe

GUAN Yong-yuan

WEI Song-jie

CHEN Rong-yuan

WEN Jie-bin

HUANG Shao-nian

HE Ye-yu

ZHANG Wei

Related Institution

School of Computer Science and Engineering, School of Cyber Science and Engineering, Nanjing University of Science and Technology

College of Frontier Intersection, Hunan University of Technology and Business

Key Laboratory of Hunan Province for Statistical Learning and Intelligent Computation, Hunan University of Technology and Business

School of Computer Science, Hunan University of Technology and Business

Faculty of Artificial Intelligence in Education, Central China Normal University

⁰

C&RM-MAKT Prediction Algorithm Using Multi-Head Attention Mechanism

DOI：10.12263/DZXB.20220790

摘要

Abstract

关键词

Keywords

references