MoGE：基于图上下文增强的多任务推荐算法

周俊; 胡斌斌; 张志强; 陈超超

doi:10.12263/DZXB.20220964

您当前的位置：

首页 >

文章列表页 >

MoGE：基于图上下文增强的多任务推荐算法

学术论文 | 更新时间：2025-12-08

- MoGE：基于图上下文增强的多任务推荐算法
- MoGE: Graph Context Enhanced Multi-Task Recommendation Method
- 电子学报 2023年51卷第11期页码：3377-3387
- 作者机构：
  
  1.浙江大学计算机科学与技术学院，浙江杭州 310027
  2.蚂蚁科技集团股份有限公司机器智能部，浙江杭州 310000
- 作者简介：
  
  [ "周俊男，1986年出生，湖南人.浙江大学博士.蚂蚁科技集团股份有限公司机器智能部负责人.主要研究方向为机器学习、图神经网络等.中国电子学会会员编号：E190028162S. E-mail: jun.zhoujun@antfin.com" ]
  [ "陈超超（通讯作者）男，1988年出生，河南人.浙江大学特聘研究员.主要研究方向为图神经网络、推荐系统等.中国电子学会会员编号：E190025053M. E-mail: zjuccc@zju.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(72192823;62172362)
- DOI：10.12263/DZXB.20220964
  中图分类号： TP183;
- 收稿：2022-08-16，
  
  修回：2022-09-21，
  
  纸质出版：2023-11-25
- 稿件说明：
移动端阅览
周俊,胡斌斌,张志强等.MoGE：基于图上下文增强的多任务推荐算法[J].电子学报,2023,51(11):3377-3387.

ZHOU Jun,HU Bin-bin,ZHANG Zhi-qiang,et al.MoGE: Graph Context Enhanced Multi-Task Recommendation Method[J].ACTA ELECTRONICA SINICA,2023,51(11):3377-3387.
周俊,胡斌斌,张志强等.MoGE：基于图上下文增强的多任务推荐算法[J].电子学报,2023,51(11):3377-3387. DOI： 10.12263/DZXB.20220964.

ZHOU Jun,HU Bin-bin,ZHANG Zhi-qiang,et al.MoGE: Graph Context Enhanced Multi-Task Recommendation Method[J].ACTA ELECTRONICA SINICA,2023,51(11):3377-3387. DOI： 10.12263/DZXB.20220964.

摘要

多任务学习（Multi-Task Learning，MTL）通过信息共享来共同处理多个任务，已被广泛应用于大量推荐任务中.目前针对推荐的多任务学习方法，主要集中在基于共享输入特征（即描述用户-商品交互信息的特征工程）的多门控混合专家网络（Multi-gate Mixture-of-Experts，MMoE），以此来学习不同任务间的关联.最近的一些工作表明，图神经网络（Graph Neural Network，GNN）作为表征深度交互上下文的强大工具，被应用于推荐任务中，可极大地缓解在线个性化推荐服务中的数据稀疏问题.因此，我们通过设计混合图增强专家网络（Mixture of Graph enhanced Expert networks，MoGE），首次探索了用于多任务推荐的图神经网络结构.具体地说，我们提出了一种新的多通道图神经网络，利用用户-商品二部图，以及衍生的用户和商品的协同相似图来联合建模用户-商品的高阶交互信息.在学习到的深层次交互上下文的基础上，引入了一组图增强的专家网络，以协作的方式实现多任务推荐.在3个真实数据集上的实验结果表明，MoGE在所有目标任务上都持续且显著地优于最优的基线.

Abstract

Multi-task learning (MTL)

which jointly tackles multiple tasks through information sharing

has been extensively used in a variety of recommendation applications. Recently

current efforts targeted for recommendation focus on learning task relationships based on the multi-gate mixture-of-experts (MMoE) architecture with shared input features (i.e.

subtle feature engineering for user-item interaction). Recent evidences suggest the graph neural network (GNN) as a powerful component in characterizing deep interaction context for recommendation

greatly contributing to easing the data sparseness issue in online advertising services. Hence

we make the first attempt to explore the GNN towards multi-task recommendation

by designing mixture of graph enhanced expert networks (MoGE). Specifically

we propose a novel multi-channel graph neural network to jointly model high-order information with the user-item bipartite graph as well as derived collaborative similarity graphs for users and items. A group of graph augmented expert networks are introduced on top of the learnt deep interaction context for cooperatively contributing to the multi-task recommendation. Experimental results on three real-world datasets show that MoGE consistently and significantly outperforms state-of-the-art baselines across all target tasks.

关键词

Keywords

references

ZHANG Y , YANG Q . A survey on multi-task learning [J ] . IEEE Transactions on Knowledge and Data Engineering , 2022 , 34 ( 12 ): 5586 - 5609 .

MISRA I , SHRIVASTAVA A , GUPTA A , et al . Cross-stitch networks for multi-task learning [C ] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2016 : 3994 - 4003 .

MA X , ZHAO L Q , HUANG G , et al . Entire space multi-task model: An effective approach for estimating post-click conversion rate [C ] // SIGIR'18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval . New York : ACM , 2018 : 1137 - 1140 .

TANG H Y , LIU J N , ZHAO M , et al . Progressive layered extraction (PLE): A novel multi-task learning (MTL) model for personalized recommendations [C ] // Proceedings of the 14th ACM Conference on Recommender Systems . New York : ACM , 2020 : 269 - 278 .

郭景峰 , 朱晓松 , 李爽 . 直播电视推荐系统的评分预测算法研究 [J ] . 电子学报 , 2020 , 48 ( 9 ): 1735 - 1740 .

GUO J F , ZHU X S , LI S . Research on rating prediction algorithm of live TV recommender systems [J ] . Acta Electronica Sinica , 2020 , 48 ( 9 ): 1735 - 1740 . (in Chinese)

李琳 , 唐守廉 . 基于多层注意力表示的音乐推荐模型 [J ] . 电子学报 , 2020 , 48 ( 9 ): 1672 - 1679 .

LI L , TANG S L . Hierarchical attention representation model for music recommendation [J ] . Acta Electronica Sinica , 2020 , 48 ( 9 ): 1672 - 1679 . (in Chinese)

方晨 , 张恒巍 , 王娜 , 等 . 基于随机游走和多样性图排序的个性化服务推荐方法 [J ] . 电子学报 , 2018 , 46 ( 11 ): 2773 - 2780 .

FANG C , ZHANG H W , WANG N , et al . Personalized service recommendation method based on random walk and diversified graph ranking [J ] . Acta Electronica Sinica , 2018 , 46 ( 11 ): 2773 - 2780 . (in Chinese)

CHEN L , WU L , HONG R C , et al . Revisiting graph based collaborative filtering: A linear residual graph convolutional network approach [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 1 ): 27 - 34 .

WU L , SUN P J , FU Y J , et al . A neural influence diffusion model for social recommendation [C ] // Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval . New York : ACM , 2019 : 235 - 244 .

WEN H , ZHANG J , WANG Y , et al . Entire space multi-task modeling via post-click behavior decomposition for conversion rate prediction [C ] // Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval . New York : ACM , 2020 : 2377 - 2386 .

GAO C , HE X N , GAN D H , et al . Neural multi-task recommendation from multi-behavior data [C ] // 2019 IEEE 35th International Conference on Data Engineering (ICDE) . Piscataway : IEEE , 2019 : 1554 - 1557 .

LI D C , LI X , WANG J , et al . Video recommendation with multi-gate mixture of experts soft actor critic [C ] // Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval . New York : ACM , 2020 : 1553 - 1556 .

KIM S , AHN N , SOHN K A . Restoring spatially-heterogeneous distortions using mixture of experts network [C ] // Asian Conference on Computer Vision . Cham : Springer , 2021 : 185 - 201 .

WANG X , YU F , DUNLAP L , et al . Deep mixture of experts via shallow embedding [EB/OL ] . ( 2018 )[2022 ] . https://arxiv.org/abs/1806.01531.pdf https://arxiv.org/abs/1806.01531.pdf .

MA J Q , ZHAO Z , YI X Y , et al . Modeling task relationships in multi-task learning with multi-gate mixture-of-experts [C ] // Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . New York : ACM , 2018 : 1930 - 1939 .

MA J Q , ZHAO Z , CHEN J L , et al . SNR: Sub-network routing for flexible parameter sharing in multi-task learning [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2019 , 33 ( 1 ): 216 - 223 .

HUAN Z X , ZHANG G D , ZHANG X L , et al . An industrial framework for cold-start recommendation in zero-shot scenarios [C ] // Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval . New York : ACM , 2022 : 3403 - 3407 .

WU Z H , PAN S R , CHEN F W , et al . A comprehensive survey on graph neural networks [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2021 , 32 ( 1 ): 4 - 24 .

KIPF T N , WELLING M . Semi-supervised classification with graph convolutional networks [EB/OL ] . ( 2016 )[2022 ] . https://arxiv.org/abs/1609.02907.pdf https://arxiv.org/abs/1609.02907.pdf .

VELIČKOVIĆ P , CUCURULL G , CASANOVA A , et al . Graph attention networks [EB/OL ] . ( 2017 )[2022 ] . https://arxiv.org/abs/1710.10903.pdf https://arxiv.org/abs/1710.10903.pdf .

WANG X , HE X N , WANG M , et al . Neural graph collaborative filtering [C ] // Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval . New York : ACM , 2019 : 165 - 174 .

YING R , HE R N , CHEN K F , et al . Graph convolutional neural networks for web-scale recommender systems [C ] // Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . New York : ACM , 2018 : 974 - 983 .

FAN W Q , MA Y , LI Q , et al . Graph neural networks for social recommendation [C ] // The World Wide Web Conference . New York : ACM , 2019 : 417 - 426 .

WANG X , HE X N , CAO Y X , et al . KGAT: Knowledge graph attention network for recommendation [C ] // Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . New York : ACM , 2019 : 950 - 958 .

VANDENHENDE S , GEORGOULIS S , VAN GANSBEKE W , et al . Multi-task learning for dense prediction tasks: A survey [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 7 ): 3614 - 3633 .

俞春花 , 刘学军 , 李斌 , 等 . 基于上下文相似度和社会网络的移动服务推荐方法 [J ] . 电子学报 , 2017 , 45 ( 6 ): 1530 - 1536 .

YU C H , LIU X J , LI B , et al . Mobile service recommendation based on context similarity and social network [J ] . Acta Electronica Sinica , 2017 , 45 ( 6 ): 1530 - 1536 . (in Chinese)

陈杰 , 刘学军 , 李斌 , 等 . 一种基于用户动态兴趣和社交网络的微博推荐方法 [J ] . 电子学报 , 2017 , 45 ( 4 ): 898 - 905 .

CHEN J , LIU X J , LI B , et al . Personalized microblogging recommendation based on dynamic interests and social networking of users [J ] . Acta Electronica Sinica , 2017 , 45 ( 4 ): 898 - 905 . (in Chinese)

肖云鹏 , 刘瀚松 , 刘宴兵 . 一种基于二部图和节点角色划分的社交网络推荐方案 [J ] . 电子学报 , 2017 , 45 ( 10 ): 2425 - 2433 .

XIAO Y P , LIU H S , LIU Y B . A social network recommendation scheme based on bipartite graph and node role division [J ] . Acta Electronica Sinica , 2017 , 45 ( 10 ): 2425 - 2433 . (in Chinese)

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all You need [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York : ACM , 2017 : 6000 - 6010 .

XU K , LI C T , TIAN Y L , et al . Representation learning on graphs with jumping knowledge networks [EB/OL ] . ( 2018 )[2022 ] . https://arxiv.org/abs/1806.03536.pdf https://arxiv.org/abs/1806.03536.pdf .

QIN Z , CHENG Y C , ZHAO Z , et al . Multitask mixture of sequential experts for user activity streams [C ] // Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . New York : ACM , 2020 : 3083 - 3091 .

KENDALL A , GAL Y . What uncertainties do we need in Bayesian deep learning for computer vision? [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York : ACM , 2017 : 5580 - 5590 .

CIPOLLA R , GAL Y , KENDALL A . Multi-task learning using uncertainty to weigh losses for scene geometry and semantics [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 7482 - 7491 .

WANG M H , LIN Y J , LIN G L , et al . M2GRL: A multi-task multi-view graph representation learning framework for web-scale recommender systems [C ] // Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . New York : ACM , 2020 : 2349 - 2358 .

HSU W T , LIN C K , LEE M Y , et al . A unified model for extractive and abstractive summarization using inconsistency loss [EB/OL ] . ( 2018 )[2022 ] . https://arxiv.org/abs/1805.06266.pdf https://arxiv.org/abs/1805.06266.pdf .

NIU X C , LI B F , LI C L , et al . Heterogeneous graph augmented multi-scenario sharing recommendation with tree-guided expert networks [C ] // Proceedings of the 14th ACM International Conference on Web Search and Data Mining . New York : ACM , 2021 : 1038 - 1046 .

LIU Z Q , CHEN C C , LI L F , et al . GeniePath: Graph neural networks with adaptive receptive paths [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2019 , 33 ( 1 ): 4424 - 4431 .

HAMILTON W L , YING R , LESKOVEC J . Inductive representation learning on large graphs [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York : ACM , 2017 : 1025 - 1035 .

CARUANA R . Multitask learning [J ] . Machine Learning , 1997 , 28 ( 1 ): 41 - 75 .

LI S H , YANG D K , ZHANG B F . MRIF: Multi-resolution interest fusion for recommendation [C ] // Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval . New York : ACM , 2020 : 1765 - 1768 .

TIAN Y , CHANG J X , NIU Y N , et al . When multi-level meets multi-interest: A multi-grained neural model for sequential recommendation [C ] // Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval . New York : ACM , 2022 : 1632 - 1641 .

LI P C , LI R Z , DA Q , et al . Improving multi-scenario learning to rank in E-commerce by exploiting task relationships in the label space [C ] // Proceedings of the 29th ACM International Conference on Information & Knowledge Management . New York : ACM , 2020 : 2605 - 2612 .

ZHOU J , LI X L , ZHAO P L , et al . KunPeng: Parameter server based distributed learning systems and its applications in alibaba and ant financial [C ] // Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . New York : ACM , 2017 : 1693 - 1702 .

ZHANG D L , HUANG X , LIU Z Q , et al . AGL: A scalable system for industrial-purpose graph machine learning [J ] . Proceedings of the VLDB Endowment , 2020 , 13 ( 12 ): 3125 - 3137 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于实时动态图联合学习框架的金融交易风控技术

基于EIMYOLO的高分遥感图像目标检测

基于多重注意力和感知加权学习的单图像高动态范围重建