FuseGate：基于置信度门控的词元级并联协作机制

田淑娟; 罗宇航; 项淑桓; 李艳春; 代星霞; 申冬苏

doi:10.12263/DZXB.20260419

您当前的位置：

首页 >

文章列表页 >

FuseGate：基于置信度门控的词元级并联协作机制

更新时间：2026-07-29

- FuseGate：基于置信度门控的词元级并联协作机制
- FuseGate: A Confidence-Gated Mechanism for Token-Level Parallel Collaboration
- 电子学报 2026年页码：1-15
- 作者机构：
  
  1.湘潭大学计算机学院，湖南湘潭 411105
  2.湖南省智慧网络国际科技创新合作基地，湖南湘潭 411105
- 作者简介：
  
  田淑娟女，1982年6月出生于湖南省郴州市。现为湘潭大学计算机学院教授、博士生导师。主要研究方向为模型互联网、网络安全和边缘计算。E-mail: sjtianwork@xtu.edu.cn
  罗宇航男，2003年3月出生于湖南省岳阳市。现为湘潭大学计算机学院硕士研究生。主要研究方向为模型互联网、智能体和大小模型协同。E-mail: 18075120223@163.com
  项淑桓男，2001年9月出生于江西省景德镇市。现为湘潭大学计算机学院硕士研究生。主要研究方向为联邦学习、大模型和人工智能安全。E-mail: 202531630501@smail.xtu.edu.cn
  李艳春女，1981年2月出生于湖南省涟源市。现为湘潭大学计算机学院副教授、硕士生导师。主要研究方向为图像处理、计算机视觉和深度模型安全。E-mail: ycli@xtu.edu.cn
  代星霞女，1996年2月出生于湖南省长沙市。现为湘潭大学计算机学院讲师、硕士生导师。主要研究方向为边缘计算、大小模型协同和智能优化。E-mail: xingxdai@xtu.edu.cn
  申冬苏女，1978年11月出生于湖南省邵阳市。现为湘潭大学计算机学院副教授、硕士生导师。主要研究方向为网络安全、密码学和人工智能安全。E-mail: dsshen@xtu.edu.cn
- 基金信息：
  
  国家自然科学基金(U24A20247;62172349;62372396;6250073741);湖南省重大标志性创新示范工程项目(2022XK2301)
- DOI：10.12263/DZXB.20260419
  中图分类号： TN92;
- 收稿：2026-03-31，
  
  录用：2026-07-02，
  
  网络首发：2026-07-29，
- 稿件说明：
移动端阅览
田淑娟, 罗宇航, 项淑桓, 等. FuseGate：基于置信度门控的词元级并联协作机制[J/OL]. 电子学报, 2026,1-15.

TIAN Shujuan, LUO Yuhang, XIANG Shuhuan, et al. FuseGate: A Confidence-Gated Mechanism for Token-Level Parallel Collaboration[J/OL]. ACTA ELECTRONICA SINICA, 2026, 1-15.
田淑娟, 罗宇航, 项淑桓, 等. FuseGate：基于置信度门控的词元级并联协作机制[J/OL]. 电子学报, 2026,1-15. DOI： 10.12263/DZXB.20260419.

TIAN Shujuan, LUO Yuhang, XIANG Shuhuan, et al. FuseGate: A Confidence-Gated Mechanism for Token-Level Parallel Collaboration[J/OL]. ACTA ELECTRONICA SINICA, 2026, 1-15. DOI： 10.12263/DZXB.20260419.

摘要

在模型互联网场景中，部署于不同边缘节点的异构模型可通过协作推理实现能力互补，从而提升系统在复杂任务上的推理能力。词元级并联协作能够在自回归解码过程中融合多个模型的预测信息，是实现细粒度模型协同推理的重要方式。现有词元级并联协作方法多采用逐词元同步的全程协作策略，即在每个解码步均调用多个模型参与候选词元生成与结果聚合。该策略虽有助于提升词元生成质量，但多模型协作并非在所有解码位置均能产生有效收益。在基模型具有较高预测置信度的词元位置继续执行多模型协作，往往会引入额外的计算开销与同步等待，从而增加单词元生成平均时延。针对上述问题，本文提出一种基于置信度门控的词元级并联协作机制FuseGate，可作为轻量级模块集成至现有词元级并联协作框架中。FuseGate无需额外预训练或模型参数更新，也不改变原有的自回归解码范式。该机制以基模型当前解码步Top-1与Top-2候选词元的logits差值表征预测置信度，并利用滑动窗口记录近期置信度变化，根据历史置信度分布动态调整协作判定阈值。在此基础上，FuseGate判断当前解码位置是否需要触发多模型协作，从而将协作计算优先分配至低置信度位置，减少全程协作产生的冗余计算与同步开销。本文构建多组异构模型组合，并在计算推理、阅读理解、多步数学推理和跨领域知识理解等多类任务上开展实验。实验结果表明，FuseGate能够适配不同类型的词元级并联协作框架，在整体推理准确率与全程协作基本相当的情况下，将单词元生成平均时延降低约46％。进一步分析表明，FuseGate对具有不同能力差距的模型组合均表现出较好的适用性，能够在减少无效协作开销的同时，保留有效协作带来的推理性能增益。研究结果表明，基于置信度的选择性协作能够在推理准确率与生成效率之间取得更优折中，为模型互联网场景下异构模型的高效词元级协同推理提供一种轻量化且具有良好框架兼容性的协作机制。

Abstract

In the AI-Model Network

heterogeneous models deployed at different edge nodes can complement one another through collaborative inference

thereby enhancing the system’s reasoning capability on complex tasks. Token-level parallel collaboration enables fine-grained model cooperation by combining predictions from multiple models during autoregressive decoding. Existing methods commonly adopt full token-wise synchronous collaboration

in which multiple models are invoked at every decoding step for candidate token generation and result aggregation. Although this strategy can improve token generation quality

multi-model collaboration does not yield meaningful benefits at every decoding position. When the base model already has high confidence in its prediction

continuing to involve additional models introduces unnecessary computation and synchronization delays

increasing the average per-token generation latency. To address this problem

we propose FuseGate

a confidence-gated mechanism for token-level parallel collaboration that can be integrated into existing collaboration frameworks as a lightweight module. FuseGate requires neither additional pretraining nor model parameter updates and does not alter the original autoregressive decoding paradigm. At each decoding step

it measures the prediction confidence of the base model using the logit difference between its Top-1 and Top-2 candidate tokens

records recent confidence variations through a sliding window

and dynamically adjusts the collaboration threshold according to the historical confidence distribution. Based on this adaptive threshold

FuseGate determines whether multi-model collaboration should be triggered at the current decoding position

thereby concentrating collaborative computation on low-confidence positions and reducing the redundant computation and synchronization overhead caused by full collaboration. We evaluate FuseGate using multiple heterogeneous model combinations on tasks including computational reasoning

reading comprehension

multi-step mathematical reasoning

and cross-domain knowledge understanding. Experimental results show that FuseGate is compatible with different token-level parallel collaboration frameworks and reduces the average per-token generation latency by approximately 46% while maintaining overall inference accuracy comparable to that of full collaboration. Further analysis shows that FuseGate remains effective across model combinations with different capability gaps

reducing ineffective collaboration while preserving the reasoning gains brought by beneficial model cooperation. These results demonstrate that confidence-based selective collaboration achieves a more favorable trade-off between inference accuracy and generation efficiency

providing a lightweight and framework-compatible mechanism for efficient token-level collaborative inference among heterogeneous models in the AI-Model Network.

关键词

Keywords

references

李哲涛 , 曾曦玉 , 王建辉 , 等 . 模型互联网: 概念、现状和未来 [J ] . 计算机研究与发展 , 2026 , 63 ( 5 ): 1305 - 1318 . DOI: 10.7544/issn1000-1239.202550223 http://dx.doi.org/10.7544/issn1000-1239.202550223 .

Li Zhetao , Zeng Xiyu , Wang Jianhui , et al . AI-model network: Concept, current state and future [J ] . Journal of Computer Research and Development , 2026 , 63 ( 5 ): 1305 - 1318 . DOI: 10.7544/issn1000-1239.202550223. http://dx.doi.org/10.7544/issn1000-1239.202550223. (in Chinese)

Bommasani R , Liang P , Lee T . Holistic evaluation of language models [J ] . Annals of the New York Academy of Sciences , 2023 , 1525 ( 1 ): 140 - 146 . DOI: 10.1111/nyas.15007 http://dx.doi.org/10.1111/nyas.15007 .

Yin Shukang , Fu Chaoyou , Zhao Sirui , et al . Woodpecker: Hallucination correction for multimodal large language models [J ] . Science China Information Sciences , 2024 , 67 ( 12 ): 220105 . DOI: 10.1007/s11432-024-4251-x http://dx.doi.org/10.1007/s11432-024-4251-x .

Chiang W L , Zheng Lianmin , Sheng Ying , et al . Chatbot arena: An open platform for evaluating LLMs by human preference [C/OL ] // Proceedings of the 41st International Conference on Machine Learning , 2024 : 8359 - 8388 . https://openreview.net/forum?id=3MW8GKNyzI https://openreview.net/forum?id=3MW8GKNyzI .

Liu Xiao , Yu Hao , Zhang Hanchen , et al . AgentBench: Evaluating LLMs as agents [C/OL ] // Proceedings of the Twelfth International Conference on Learning Representations , 2024 . https://openreview.net/forum?id=zAdUB0aCTQ https://openreview.net/forum?id=zAdUB0aCTQ .

王建辉 , 李哲涛 , 伍涛 , 等 . Token级多模型并联协作推理 [J ] . 计算机学报 , 2025 , 48 ( 11 ): 2579 - 2593 .

Wang Jianhui , Li Zhetao , Wu Tao , et al . Token-level collaborative reasoning for parallel multi-models [J ] . Chinese Journal of Computers , 2025 , 48 ( 11 ): 2579 - 2593 . (in Chinese)

Wang Junlin , Wang Jue , Athiwaratkun B , et al . Mixture-of-agents enhances large language model capabilities [C/OL ] // Proceedings of the 13th International Conference on Learning Representations , 2025 . https://openreview.net/forum?id=h0ZfDIrj7T https://openreview.net/forum?id=h0ZfDIrj7T .

Sun Qiushi , Yin Zhangyue , Li Xiang , et al . Corex: Pushing the boundaries of complex reasoning through multi-model collaboration [C/OL ] // Proceedings of Conference on Language Modeling , 2024 . https://openreview.net/forum?id=GDdxmymrwL https://openreview.net/forum?id=GDdxmymrwL .

Vaswani A , Shazeer N , Parmar N , et al . Attention is all you need [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York : Curran Associates Inc. , 2017 : 6000 - 6010 .

Cobbe K , Kosaraju V , Bavarian M , et al . Training verifiers to solve math word problems [PP/OL ] . V2. arXiv ( 2021-11-18 )[ 2026-07-12 ] . https://arxiv.org/abs/2110.14168 https://arxiv.org/abs/2110.14168 .

Chen Lingjiao , Zaharia M , Zou J . FrugalGPT: How to use large language models while reducing cost and improving performance [PP/OL ] . V1. arXiv ( 2023-05-09 )[ 2026-07-12 ] . https://arxiv.org/abs/2305.05176 https://arxiv.org/abs/2305.05176 .

Lu Keming , Yuan Hongyi , Lin Runji , et al . Routing to the expert: Efficient reward-guided ensemble of large language models [C ] // Proceedings of 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . New York : Association for Computational Linguistics , 2024 : 1964 - 1974 . DOI: 10.18653/v1/2024.naacl-long.109 http://dx.doi.org/10.18653/v1/2024.naacl-long.109 .

Pan Jiabao , Zhang Yan , Zhang Chen , et al . DynaThink: Fast or slow? A dynamic decision-making framework for large language models [C ] // Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing . New York : Association for Computational Linguistics , 2024 : 14686 - 14695 . DOI: 10.18653/v1/2024.emnlp-main.814 http://dx.doi.org/10.18653/v1/2024.emnlp-main.814 .

刘忠仁 , 李哲涛 , 王建辉 , 等 . 模型互联中多模型串并联协作推理 [J ] . 电子学报 , 2025 , 53 ( 11 ): 3817 - 3835 . DOI: 10.12263/DZXB.20250503 http://dx.doi.org/10.12263/DZXB.20250503 .

Liu Zhongren , Li Zhetao , Wang Jianhui , et al . Multi-model serial and parallel collaborative inference in AI-ModelNet [J ] . Acta Electronica Sinica , 2025 , 53 ( 11 ): 3817 - 3835 . DOI: 10.12263/DZXB.20250503. http://dx.doi.org/10.12263/DZXB.20250503. (in Chinese)

Mohammadshahi A , Shaikh A R , Yazdani M . Routoo: Learning to route to large language models effectively [PP/OL ] . V3. arXiv ( 2024-10-02 )[ 2026-07-12 ] . https://arxiv.org/abs/2401.13979 https://arxiv.org/abs/2401.13979 .

Yu Y C , Kuo C C , Ye Ziqi , et al . Breaking the ceiling of the LLM community by treating token generation as a classification for ensembling [C ] // Findings of the Association for Computational Linguistics: EMNLP 2024 . New York : Association for Computational Linguistics , 2024 : 1826 - 1839 . DOI: 10.18653/v1/2024.findings-emnlp.99 http://dx.doi.org/10.18653/v1/2024.findings-emnlp.99 .

Yao Yuxuan , Wu Han , Liu Mingyang , et al . Determine-then-ensemble: Necessity of top-k union for large language model ensembling [C/OL ] // Proceedings of the 13th International Conference on Learning Representations , 2025 . https://openreview.net/forum?id=FDnZFpHmU4 https://openreview.net/forum?id=FDnZFpHmU4 .

Hao Chao , Wang Zezheng , Huang Yanhua , et al . Token-by-token election: Improving language model reasoning through token-level multi-model collaboration [EB/OL ] . ( 2024-09-13 )[ 2026-07-12 ] . https://openreview.net/forum?id=QPZy2XMgzn https://openreview.net/forum?id=QPZy2XMgzn .

Xu Yangyifan , Ren Shuo , Zhang Jiajun . Collaborative beam search: Enhancing LLM reasoning via collective consensus [C ] // Proceedings of 2025 Conference on Empirical Methods in Natural Language Processing . New York : Association for Computational Linguistics , 2025 : 11398 - 11410 . DOI: 10.18653/v1/2025.emnlp-main.574 http://dx.doi.org/10.18653/v1/2025.emnlp-main.574 .

Huang Yichong , Feng Xiaocheng , Li Baohang , et al . Ensemble learning for heterogeneous large language models with deep parallel collaboration [C ] // Proceedings of the 38th International Conference on Neural Information Processing Systems . New York : Curran Associates Inc. , 2024 : 3808 . DOI: 10.52202/079017-3808 http://dx.doi.org/10.52202/079017-3808 .

Bi Xiao , Chen Deli , Chen Guanting , et al . DeepSeek LLM: Scaling open-source language models with longtermism [PP/OL ] . V1. arXiv ( 2024-01-05 )[ 2026-07-12 ] . https://arxiv.org/abs/2401.02954 https://arxiv.org/abs/2401.02954 .

QWEN Team . Introducing Qwen1.5 [EB/OL ] . ( 2024-02-04 )[ 2026-07-12 ] . https://qwenlm.github.io/blog/qwen1.5/ https://qwenlm.github.io/blog/qwen1.5/ .

Mistral AI Team . Mistral 7 B[EB/OL ] . ( 2023-09-27 )[ 2026-07-12 ] . https://mistral.ai/news/announcing-mistral-7b/ https://mistral.ai/news/announcing-mistral-7b/ .

Grattafiori A , Dubey A , Jauhri A , et al . The llama 3 herd of models [PP/OL ] . V3. arXiv ( 2024-11-23 )[ 2026-07-12 ] . https://arxiv.org/abs/2407.21783 https://arxiv.org/abs/2407.21783 .

Tencent . Hunyuan-7B-Instruct [EB/OL ] . ( 2025-07-30 )[ 2026-07-12 ] . https://huggingface.co/tencent/Hunyuan-7B-Instruct/ https://huggingface.co/tencent/Hunyuan-7B-Instruct/ .

Du Yilun , Li Shuang , Torralba A , et al . Improving factuality and reasoning in language models through multiagent debate [C/OL ] // Proceedings of the 41st International Conference on Machine Learning , 2024 : 467 . https://openreview.net/challenge?redirect=%2Fpdf%3Fid%3Dzj7YuTE4t8 https://openreview.net/challenge?redirect=%2Fpdf%3Fid%3Dzj7YuTE4t8 .

Clark C , Lee K , Chang Mingwei , et al . BoolQ: Exploring the surprising difficulty of natural yes/no questions [C ] // Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . New York : Association for Computational Linguistics , 2019 : 2924 - 2936 . DOI: 10.18653/v1/N19-1300 http://dx.doi.org/10.18653/v1/N19-1300 .

Hendrycks D , Burns C , Basart S , et al . Measuring massive multitask language understanding [C/OL ] // Proceedings of the 9th International Conference on Learning Representations , 2021 . https://openreview.net/forum?id=d7KBjmI3GmQ https://openreview.net/forum?id=d7KBjmI3GmQ .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

大语言模型在电子设计自动化中的应用综述

模型互联网中以共识驱动的大模型引导推理

基于大语言模型的加密流量分析方法研究综述

MASTER：基于知识注入与意图强化的大语言模型漏洞验证代码生成框架

一种融合双粒度知识库的仿生设计方案智能生成方法