基于内在质量约束的文本生成和评价综述

兰玉乾; 饶元; 李冠呈; 孙菱; 夏昺灿; 辛婷婷

doi:10.12263/DZXB.20230826

您当前的位置：

首页 >

文章列表页 >

基于内在质量约束的文本生成和评价综述

综述评论 | 更新时间：2025-12-11

- 基于内在质量约束的文本生成和评价综述
- A Survey of Text Generation and Evaluation Based on Intrinsic Quality Constraints
- 电子学报 2024年52卷第2期页码：633-659
- 作者机构：
  
  1.西安交通大学软件学院社会智能与复杂数据处理实验室，陕西西安 710049
  2.中国长峰机电技术研究设计院，北京 100854
- 作者简介：
  
  [ "兰玉乾男，1990年4月出生于陕西省西安市.2017年毕业于西北工业大学软件学院软件工程专业.现为西安交通大学软件学院博士研究生.主要研究方向为自然语言处理、文本生成、对话系统和社会智能等. E-mail: Yuqian_Lan_xjtu@stu.xjtu.edu.cn" ]
  [ "饶元男，1973年2月出生于湖北省武汉市.现为西安交通大学电信学部软件学院教授、博士生导师.陕西省人工智能联合（重点）实验室秘书长、副主任，西安市社会智能与复杂数据处理重点实验室主任.获得军委军事科学与技术进步奖、中国发明协会、陕西省教育厅高校优秀成果、王宽诚育才奖等20余项.国内外发表学术论文70余篇.申请软件著作权30余项，专利24项.主要研究方向为文本数据挖掘、自然语言处理、机器学习以及社会网络分析等. E-mail: yuanrao@163.com" ]
  [ "李冠呈男，1991年9月出生于山东省日照市.2018年毕业于北京理工大学自动化学院.现为中国长峰机电技术研究设计院工程师.主要研究方向为体系设计. E-mail: 414773396@qq.com" ]
  [ "孙菱女，1997年出生于四川省广元市.2019年毕业于中央民族大学软件工程专业.现于西安交通大学软件学院攻读博士学位.主要研究方向为复杂网络和虚假信息传播.E-mail: sunling@stu.xjtu.edu.cn" ]
  [ "夏昺灿男，1997年12月出生于河南省灵宝市.2019年本科毕业于西北农林科技大学信息工程学院计算机科学与技术专业.2023年硕士毕业于西安交通大学软件学院软件工程专业.现为西安交通大学博士研究生.主要研究方向为图神经网络. E-mail: bingcan92@xjtu.stu.edu.cn" ]
  [ "辛婷婷女，1995年8月出生于山西省运城市.2021年毕业于西安工程大学计算机科学与技术学院.现为西安交通大学博士研究生.主要研究方向为社交网络中信息传播策略. E-mail: xinting828@gmail.com" ]
- 基金信息：
  
  国家自然科学基金重点项目(U22B2036);科技部重点研发计划项目(2019YFB2102300);中央高校建设世界一流大学（学科）和特色发展引导专项资金项目(PY3A022)
- DOI：10.12263/DZXB.20230826
  中图分类号： TP183;
- 收稿：2023-09-01，
  
  修回：2024-01-02，
  
  纸质出版：2024-02-25
- 稿件说明：
移动端阅览
兰玉乾,饶元,李冠呈,等.基于内在质量约束的文本生成和评价综述[J].电子学报,2024,52(02):633-659.

LAN Yu-qian, RAO Yuan, LI Guan-cheng, et al.A Survey of Text Generation and Evaluation Based on Intrinsic Quality Constraints[J].Acta Electronica Sinica, 2024, 52(02): 633-659.
兰玉乾,饶元,李冠呈,等.基于内在质量约束的文本生成和评价综述[J].电子学报,2024,52(02):633-659. DOI：10.12263/DZXB.20230826

LAN Yu-qian, RAO Yuan, LI Guan-cheng, et al.A Survey of Text Generation and Evaluation Based on Intrinsic Quality Constraints[J].Acta Electronica Sinica, 2024, 52(02): 633-659. DOI：10.12263/DZXB.20230826

摘要

近年来，以ChatGPT为代表的能够适应复杂场景、并能满足人类的各种应用需求为目标的文本生成算法模型成为学术界与产业界共同关注的焦点.然而，ChatGPT等大规模语言模型（Large Language Model，LLM）高度忠实于用户意图的优势隐含了部分的事实性错误，而且也需要依靠提示内容来控制细致的生成质量和领域适应性，因此，研究以内在质量约束为核心的文本生成方法仍具有重要意义.本文在近年来关键的内容生成模型和技术对比研究的基础上，定义了基于内在质量约束的文本生成的基本形式，以及基于“信、达、雅”的6种质量特征；针对这6种质量特征，分析并总结了生成器模型的设计和相关算法；同时，围绕不同的内在质量特征总结了多种自动评价和人工评价指标与方法.最后，本文对文本内在质量约束技术的未来研究方向进行了展望.

Abstract

Recently

the outstanding text generation language models represented by ChatGPT

which can adapt to complex scenes and meet various application demands of human beings

has become the focuses of both the academic and industrial circles. However

the advantage of large language models (LLM) such as ChatGPT that are highly faithful to user intent implies some factual errors

and it is also necessary to rely on prompt content to control the detailed generation quality and domain adaptability

so it is still of great significance to study text generation with intrinsic quality constraints as the core. Based on the comparative study of key content generation models and technologies in recent years

this paper defined the basic form of text generation with intrinsic quality constraints

and six quality features based on “credibility

expressiveness and elegance”. In view of these 6 quality features

we provided analysis and comparison of generator model design and related algorithms. Besides

various automatic and human evaluation methods for different intrinsic quality features are summarized. Finally

this paper looks forward to the future research directions of intrinsic quality constraint technology.

关键词

Keywords

references

OUYANG , WU J , JIANG X , et al . Training language models to follow instructions with human feedback [EB/OL]. ( 2022-03-04 )[ 2023-09-01 ]. https://arxiv.org/abs/2203.02155 https://arxiv.org/abs/2203.02155 .

SCHULMAN J , ZOPH B , KIM C , et al . ChatGPT: Optimizing language models for dialogue [R/OL]. ( 2022-11-30 )[ 2023-09-01 ]. https://openai.com/blog/chatgpt https://openai.com/blog/chatgpt .

BROWN T B , MANN B , RYDER N , et al . Language models are few-shot learners [C]// Proceedings of the 34th International Conference on Neural Information Processing Systems . New York : ACM , 2020 : 1877 - 1901 .

LEWIS M , LIU Y H , GOYAL N , et al . BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension [C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2020 : 7871 - 80 .

TURING A M . Computing machinery and intelligence [M]//EPSTEIN R, ROBERTS G, BEBER G. Parsing the Turing Test . Dordrecht : Springer , 2009 : 23 - 65 .

ZHANG M . An inquiry into Yan fu’s translation theory of faithfulness, expressiveness, and elegance: The beginning of China’s modern translation theory [J]. Trans-Humanities Journal , 2013 , 6 ( 3 ): 179 - 196 .

RAFFEL C , SHAZEER N , ROBERTS A , et al . Exploring the limits of transfer learning with a unified text-to-text transformer [EB/OL]. ( 2019-10-23 )[ 2023-09-01 ]. https://arxiv.org/abs/1910.10683 https://arxiv.org/abs/1910.10683 .

BUBECK S , CHANDRASEKARAN V , ELDAN R , et al . Sparks of Artificial General Intelligence: Early experiments with GPT-4 [EB/OL]. ( 2023-03-22 )[ 2023-09-01 ]. http://arxiv.org/abs/2303.12712.pdf http://arxiv.org/abs/2303.12712.pdf .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [EB/OL]. ( 2017-06-12 )[ 2023-09-01 ]. http://arxiv.org/abs/1706.03762.pdf http://arxiv.org/abs/1706.03762.pdf .

DEVLIN J , CHANG M-W , LEE K , et al . Bert: Pre-training of deep bidirectional transformers for language understanding [C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) . Minneapolis : Association for Computational Linguistics , 2019 : 4171 - 4186 .

RADFORD A , NARASIMHAN K , SALIMANS T , et al . Improving language understanding by generative pre-training [EB/OL]. ( 2018-10-11 )[ 2023-09-01 ]. https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf .

RADFORD A , WU J , CHILD R , et al . Language models are unsupervised multitask learners [EB/OL]. ( 2019-02-14 )[ 2023-09-01 ]. https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf .

LIU Y H , OTT M , GOYAL N , et al . RoBERTa: A robustly optimized BERT pretraining approach [EB/OL]. ( 2019-07-26 )[ 2023-09-01 ]. https://arxiv.org/abs/1907.11692 https://arxiv.org/abs/1907.11692 .

JOSHI M , CHEN D Q , LIU Y H , et al . SpanBERT: Improving pre-training by representing and predicting spans [J]. Transactions of the Association for Computational Linguistics , 2020 , 8 : 64 - 77 .

REIMERS N , GUREVYCH I . Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks [C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) . Stroudsburg : Association for Computational Linguistics , 2019 : 3980 - 3990 .

DAI Z H , YANG Z L , YANG Y M , et al . Transformer-XL: Attentive language models beyond a fixed-length context [C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2019 : 2978 - 2988 .

YANG Z L , DAI Z H , YANG Y M , et al . XLNet: Generalized autoregressive pretraining for language understanding [C]// NIPS’19: Proceedings of the 33rd International Conference on Neural Information Processing Systems . New York : ACM , 2019 : 5753 - 5763 .

TAN B W , YANG Z C , AI-SHEDIVAT M , et al . Progressive generation of long text with pretrained language models [EB/OL]. ( 2020-06-28 )[ 2023-09-01 ]. https://arxiv.org/abs/2006.15720 https://arxiv.org/abs/2006.15720 .

HUA X Y , WANG L . PAIR: Planning and iterative refinement in pre-trained transformers for long text generation [C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . Kerrville : Association for Computational Linguistics , 2020 : 781 - 793 .

JANG Y , LIM J , HUR Y , et al . Call for customized conversation: Customized conversation grounding persona and knowledge [J]. Proceedings of the AAAI Conference on Artificial Intelligence , 2022 , 36 ( 10 ): 10803 - 10812 .

CHEN Z , XU W D , WANG B T , et al . A blockchain-based preserving and sharing system for medical data privacy [J]. Future Generation Computer Systems , 2021 , 124 ( C ): 338 - 350 .

LE T , PARK N , LEE D . SHIELD: defending textual neural networks against multiple black-box adversarial attacks with stochastic multi-expert patcher [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2022 : 6661 - 6674 .

YI X Y , LIU Z H , LI W H , et al . Text style transfer via learning style instance supported latent space [C]// Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence . California : International Joint Conferences on Artificial Intelligence Organization , 2020 : 3801 - 3807 .

ZHOU C L , CHEN L Y , LIU J C , et al . Exploring contextual word-level style relevance for unsupervised style transfer [C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2020 : 7135 - 7144 .

MO L Z , WEI J L , HUANG Q B , et al . Incorporating sentimental trend into gated mechanism based transformer network for story ending generation [J]. Neurocomputing , 2021 , 453 ( C ): 453 - 464 .

WEN Z Y , CAO J N , YANG R S , et al . Automatically select emotion for response via personality-affected emotion transition [C]// Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 . Stroudsburg : Association for Computational Linguistics , 2021 : 5010 - 5020 .

YIN X J , WAN X J . How do Seq2Seq models perform on end-to-end data-to-text generation? [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2022 : 7701 - 7710 .

SHU C , ZHANG Y S , DONG X Y , et al . Logic-consistency text generation from semantic parses [C]// Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 . Stroudsburg : Association for Computational Linguistics , 2021 : 4414 - 4426 .

DER LEE CHRIS V , ALBERT G , EMIEL V M , et al . Human evaluation of automatically generated text: Current trends and best practice guidelines [J]. Computer Speech & Language , 2021 , 67 : 101151 .

IQBAL T , S J J O K S U-C QURESHI , SCIENCES I . The survey: Text generation models in deep learning [J]. Journal of King Saud University - Computer and Information Sciences , 2020 , 34 ( 6 ): 2515 - 2528 .

LI J Y , TANG T Y , ZHAO W X , et al . Pretrained language model for text generation: A survey [C]// Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence . California : International Joint Conferences on Artificial Intelligence Organization , 2021 : 4492 - 4499 .

CELIKYILMAZ A , CLARK E , GAO J F . Evaluation of text generation: A survey [EB/OL]. ( 2020-06-26 )[ 2023-09-01 ]. http://arxiv.org/abs/2006.14799.pdf http://arxiv.org/abs/2006.14799.pdf .

JIN D , JIN Z J , HU Z T , et al . Deep learning for text style transfer: A survey [J]. Comput Linguistics , 2022 , 48 ( 1 ): 155 - 205 .

ZHOU J N , BHAT S . Paraphrase generation: A survey of the state of the art [C]// Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . Stroudsburg : Association for Computational Linguistics , 2021 : 5075 - 5086 .

ZHAO H , PHUNG D , HUYNH V , et al . Topic modelling meets deep neural networks: A survey [C]// Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence . California : International Joint Conferences on Artificial Intelligence Organization , 2021 : 4713 - 4720 .

JI Z W , LEE N , FRIESKE R , et al . Survey of hallucination in natural language generation [J]. ACM Computing Surveys , 2022 , 55 ( 12 ): 248 .

TANG C , GUERIN F , LIN C H . Recent advances in neural text generation: A task-agnostic survey [EB/OL]. ( 2022-03-06 )[ 2023-09-01 ]. http://arxiv.org/abs/2203.03047.pdf http://arxiv.org/abs/2203.03047.pdf .

HALLIDAY M A K , MATTHIESSEN C M I M . Halliday’s Introduction to Functional Grammar [M]. London : Routledge , 2013 .

REBUFFEL C , ROBERTI M , SOULIER L , et al . Controlling hallucinations at word level in data-to-text generation [J]. Data Mining and Knowledge Discovery , 2022 , 36 ( 1 ): 318 - 354 .

RASHKIN H , REITTER D , TOMAR G S , et al . Increasing faithfulness in knowledge-grounded dialogue with controllable features [C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2021 : 704 - 718 .

LI H R , ZHU J N , ZHANG J J , et al . Ensure the correctness of the summary: Incorporate entailment knowledge into abstractive sentence summarization [C]// Proceedings of the 27th International Conference on Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2018 : 1430 - 1441 .

LIU T Y , ZHENG X , CHANG B B , et al . Towards faithfulness in open domain table-to-text generation from an entity-centric view [J]. Proceedings of the AAAI Conference on Artificial Intelligence , 2021 , 35 ( 15 ): 13415 - 13423 .

ZHANG J Q , ZHAO Y , SALEH M , et al . PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization [C]// Proceedings of the 37th International Conference on Machine Learning . New York : ACM , 2020 : 11328 - 11339 .

ARALIKATTE R , NARAYAN S , MAYNEZ J , et al . Focus attention: Promoting faithfulness and diversity in summarization [C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2021 : 6078 - 6095 .

CAO S Y , WANG L . CLIFF: contrastive learning for improving faithfulness and factuality in abstractive summarization [C]// Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . Stroudsburg : Association for Computational Linguistics , 2021 : 6633 - 6649 .

LEE S , LEE D B , HWANG S J . Contrastive learning with adversarial perturbations for conditional text generation [EB/OL]. ( 2020-12-14 )[ 2023-09-01 ]. http://arxiv.org/abs/2012.07280.pdf http://arxiv.org/abs/2012.07280.pdf .

ZHU C G , XU Y C , REN X , et al . Knowledge-augmented methods for natural language processing [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts . Stroudsburg : Association for Computational Linguistics , 2022 : 12 - 20 .

ZHOU H , YOUNG T , HUANG M L , et al . Commonsense knowledge aware conversation generation with graph attention [C]// Proceedings of the 27th International Joint Conference on Artificial Intelligence . New York : ACM , 2018 : 4623 - 4629 .

TAI Y , YANG J , LIU X M , et al . MemNet: A persistent memory network for image restoration [C]// 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2017 : 4549 - 4557 .

GUAN J , HUANG F , ZHAO Z H , et al . A knowledge-enhanced pretraining model for commonsense story generation [J]. Transactions of the Association for Computational Linguistics , 2020 , 8 : 93 - 108 .

YU C L , ZHANG H M , SONG Y Q , et al . CoCoLM: complex commonsense enhanced language model with discourse relations [C]// Findings of the Association for Computational Linguistics: ACL 2022 . Stroudsburg : Association for Computational Linguistics , 2022 : 1175 - 1187 .

MADAAN N , PADHI I , PANWAR N , et al . Generate your counterfactuals: Towards controlled counterfactual generation for text [J]. Proceedings of the AAAI Conference on Artificial Intelligence , 2021 , 35 ( 15 ): 13516 - 13524 .

PARANJAPE B , LAMM M , TENNEY I . Retrieval-guided counterfactual generation for QA [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2022 : 1670 - 1686 .

BARTOLO M , THRUSH T , JIA R , et al . Improving question answering model robustness with synthetic adversarial data generation [C]// Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . Stroudsburg : Association for Computational Linguistics , 2021 : 8830 - 8848 .

CALDERON N , BEN-DAVID E , FEDER A , et al . DoCoGen: domain counterfactual generation for low resource domain adaptation [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2022 : 7727 - 7746 .

ZHANG H N , LAN Y Y , PANG L , et al . Modeling topical relevance for multi-turn dialogue generation [C]// Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence . California : International Joint Conferences on Artificial Intelligence Organization , 2020 : 3737 - 3743 .

SERBAN I , SORDONI A , BENGIO Y , et al . Building end-to-end dialogue systems using generative hierarchical neural network models [J]. Proceedings of the AAAI Conference on Artificial Intelligence , 2016 , 30 ( 1 ): 3776 - 3784 .

HE G X , GAO Z , JIANG Z R , et al . Think beyond the word: Understanding the implied textual meaning by digesting context, local, and noise [C]// Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval . New York : ACM , 2020 : 2297 - 2306 .

QIN L H , GALLEY M , BROCKETT C , et al . Conversing by reading: Contentful neural conversation with on-demand machine reading [C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2019 : 5427 - 5436 .

LIU X D , SHEN Y L , DUH K , et al . Stochastic answer networks for machine reading comprehension [C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2018 : 1694 - 1704 .

TIAN Z L , BI W , LEE D , et al . Response-anticipated memory for on-demand knowledge integration in response generation [C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2020 : 650 - 659 .

XU J , LEI Z Y , WANG H F , et al . Discovering dialog structure graph for coherent dialog generation [C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2021 : 1726 - 1739 .

PENG B L , LI C Y , LI J C , et al . SOLOIST: Few-shot task-oriented dialog with A single pre-trained auto-regressive model [EB/OL]. ( 2020-05-11 )[ 2023-09-01 ]. http://arxiv.org/abs/2005.05298.pdf http://arxiv.org/abs/2005.05298.pdf .

MORYOSSEF A , GOLDBERG Y , DAGAN I . Step-by-step: Separating planning from realization in neural data-to-text generation [EB/OL]. ( 2019-08-06 )[ 2023-09-01 ]. http://arxiv.org/abs/1904.03396.pdf http://arxiv.org/abs/1904.03396.pdf .

HUA X Y , HU Z , WANG L . Argument generation with retrieval, planning, and realization [C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2019 : 2661 - 1672 .

MARCHENKO O O , RADYVONENKO O S , IGNATOVA T S , et al . Improving text generation through introducing coherence metrics [J]. Cybernetics and Systems Analysis , 2020 , 56 ( 1 ): 13 - 21 .

WEI J , WANG X Z , SCHUURMANS D , et al . Chain-of-thought prompting elicits reasoning in large language models [EB/OL]. ( 2022-01-28 )[ 2023-09-01 ]. https://arxiv.org/abs/2201.11903 https://arxiv.org/abs/2201.11903 .

YAO S Y , YU D , ZHAO J , et al . Tree of thoughts: Deliberate problem solving with large language models [EB/OL]. ( 2023-05-17 )[ 2023-09-01 ]. http://arxiv.org/abs/2305.10601.pdf http://arxiv.org/abs/2305.10601.pdf .

CAO Y , BI W , FANG M , et al . A model-agnostic data manipulation method for persona-based dialogue generation [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2022 : 7984 - 8002 .

KIM M , KWAK B W , KIM Y , et al . Dual task framework for improving persona-grounded dialogue dataset [J]. Proceedings of the AAAI Conference on Artificial Intelligence , 2022 , 36 ( 10 ): 10912 - 10920 .

FU T C , ZHAO X L , TAO C Y , et al . There are a thousand hamlets in a thousand People's eyes: Enhancing knowledge-grounded dialogue with personal memory [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2022 : 3901 - 3913 .

YANG R Z , CHEN J X , NARASIMHAN K . Improving dialog systems for negotiation with personality modeling [C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2021 : 681 - 693 .

SONG H Y , WANG Y , ZHANG K Y , et al . BoB: BERT over BERT for training persona-based dialogue models from limited personalized data [C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2021 : 167 - 177 .

GU J C , TAO C Y , LING Z H , et al . MPC-BERT: A pre-trained language model for multi-party conversation understanding [C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2021 : 3682 - 3692 .

PARK J S , O'BRIEN J , CAI C J , et al . Generative agents: Interactive simulacra of human behavior [C]// Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology . New York : ACM , 2023 : 1 - 22 .

WANG Z L , CHIU Y Y , CHIU Y C . Humanoid agents: Platform for simulating human-like generative agents [C]// Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations . Stroudsburg : Association for Computational Linguistics , 2023 : 167 - 176 .

SCLAR M , KUMAR S , WEST P , et al . Minding language models' (lack of) theory of mind: A plug-and-play multi-character belief tracker [C]// Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2023 : 13960 - 13980 .

LEE J . Stable style transformer: Delete and generate approach with encoder-decoder for text style transfer [C]// Proceedings of the 13th International Conference on Natural Language Generation . Stroudsburg : Association for Computational Linguistics , 2020 : 195 - 204 .

SHU L , PAPANGELIS A , WANG Y C , et al . Controllable text generation with focused variation [C]// Findings of the Association for Computational Linguistics: EMNLP 2020 . Stroudsburg : Association for Computational Linguistics , 2020 : 3805 - 3817 .

SOHN K , YAN X C , LEE H . Learning structured output representation using deep conditional generative models [J]. Advances in Neural Information Processing Systems , 2015 , 2015-January: 3483- 3491 .

DAI N , LIANG J Z , QIU X P , et al . Style transformer: Unpaired text style transfer without disentangled latent representation [C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2019 : 5997 - 6007 .

CHENG Y , GAN Z , ZHANG Y Z , et al . Contextual text style transfer [C]// Findings of the Association for Computational Linguistics: EMNLP 2020 . Stroudsburg : Association for Computational Linguistics , 2020 : 2915 - 2924 .

KHAYRALLAH H , SEDOC J . SMRT chatbots: Improving non-task-oriented dialog with simulated multiple reference training [C]// Findings of the Association for Computational Linguistics: EMNLP 2020 . Stroudsburg : Association for Computational Linguistics , 2020 : 4489 - 4505 .

SU H , SHEN X Y , ZHAO S Q , et al . Diversifying dialogue generation with non-conversational text [C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2020 : 7087 - 7097 .

HU Z T , YANG Z C , LIANG X D , et al . Toward controlled generation of text [C]// Proceedings of the 34th International Conference on Machine Learning - Volume 70 . New York : ACM , 2017 : 1587 - 1596 .

KINGMA D P , WELLING M . Auto-encoding variational Bayes [EB/OL]. ( 2013-12-20 )[ 2023-09-01 ]. http://arxiv.org/abs/1312.6114.pdf http://arxiv.org/abs/1312.6114.pdf .

LACHAUX M A , JOULIN A , LAMPLE G . Target conditioning for one-to-many generation [C]// Findings of the Association for Computational Linguistics: EMNLP 2020 . Stroudsburg : Association for Computational Linguistics , 2020 : 2853 - 2862 .

CUI Z , LI Y R , ZHANG J Y , et al . Focus-constrained attention mechanism for CVAE-based response generation [C]// Findings of the Association for Computational Linguistics: EMNLP 2020 . Stroudsburg : Association for Computational Linguistics , 2020 : 2021 - 2030 .

WANG Z , RAO S W , ZHANG J , et al . Diversify question generation with continuous content selectors and question type modeling [C]// Findings of the Association for Computational Linguistics: EMNLP 2020 . Stroudsburg : Association for Computational Linguistics , 2020 : 2134 - 2143 .

SUN H , LIN Z R , ZHENG C J , et al . PsyQA: A Chinese dataset for generating long counseling text for mental health support [C]// Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 . Stroudsburg : Association for Computational Linguistics , 2021 : 1489 - 1503 .

XU J , SZLAM A , WESTON J . Beyond goldfish memory: Long-term open-domain conversation [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2022 : 5180 - 5197 .

GU N L , ASH E , HAHNLOSER R . MemSum: extractive summarization of long documents using multi-step episodic Markov decision processes [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2022 : 6507 - 6522 .

PAPINENI K , ROUKOS S , WARD T , et al . BLEU: A method for automatic evaluation of machine translation [C]// Proceedings of the 40th Annual Meeting on Association for Computational Linguistics . New York : ACM , 2002 : 311 - 318 .

FABBRI A , WU C S , LIU W H , et al . QAFactEval: improved QA-based factual consistency evaluation for summarization [C]// Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . Stroudsburg : Association for Computational Linguistics , 2022 : 2587 - 2601 .

DHINGRA B , FARUQUI M , PARIKH A , et al . Handling divergent reference texts when evaluating table-to-text generation [C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2019 : 4884 - 4895 .

DURMUS E , HE H , DIAB M . FEQA: A question answering evaluation framework for faithfulness assessment in abstractive summarization [C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2020 : 5055 - 5070 .

SCIALOM T , DRAY P A , LAMPRIER S , et al . QuestEval: summarization asks for fact-based evaluation [C]// Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . Stroudsburg : Association for Computational Linguistics , 2021 : 6594 - 6604 .

FALKE T , RIBEIRO L F R , UTAMA P A , et al . Ranking generated summaries by correctness: An interesting but challenging application for natural language inference [C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2019 : 2214 - 2220 .

MAYNEZ J , NARAYAN S , BOHNET B , et al . On faithfulness and factuality in abstractive summarization [C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2020 : 1906 - 1919 .

NIE Y X , WILLIAMSON M , BANSAL M , et al . I like fish, especially dolphins: Addressing Contradictions in Dialogue Modeling [C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2021 : 1699 - 1713 .

GOYAL T , DURRETT G . Evaluating factuality in generation with dependency-level entailment [C]// Findings of the Association for Computational Linguistics: EMNLP 2020 . Stroudsburg : Association for Computational Linguistics , 2020 : 3592 - 3603 .

KRYSCINSKI W , MCCANN B , XIONG C M , et al . Evaluating the factual consistency of abstractive text summarization [C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . Stroudsburg : Association for Computational Linguistics , 2020 : 9332 - 9346 .

DZIRI N , RASHKIN H , LINZEN T , et al . Evaluating groundedness in dialogue systems: The BEGIN benchmark [EB/OL]. ( 2021-04-30 )[ 2023-09-01 ]. http://arxiv.org/abs/2105.00071.pdf http://arxiv.org/abs/2105.00071.pdf .

WANG A , CHO K , LEWIS M . Asking and answering questions to evaluate the factual consistency of summaries [C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2020 : 5008 - 5020 .

NAN F , NOGUEIRA DOS SANTOS C , ZHU H H , et al . Improving factual consistency of abstractive summarization via question answering [C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2021 : 6881 - 6894 .

QIN L B , XIE T B , HUANG S J , et al . Don't be contradicted with anything! CI-ToD: Towards benchmarking consistency for task-oriented dialogue system [C]// Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . Stroudsburg : Association for Computational Linguistics , 2021 : 2357 - 2367 .

GUPTA P , WU C S , LIU W H , et al . DialFact: A benchmark for fact-checking in dialogue [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Stroudsburg : Association for Computational Linguistics , 2022 : 3785 - 3801 .

LEE N , PING W , XU P , et al . Factuality enhanced language models for open-ended text generation [EB/OL]. ( 2022-06-09 )[ 2023-09-01 ]. http://arxiv.org/abs/2206.04624.pdf http://arxiv.org/abs/2206.04624.pdf .

LIN C Y . ROUGE: A package for automatic evaluation of summaries [C]// Proceedings of the Text Summarization Branches Out . Stroudsburg : Association for Computational Linguistics , 2004 : 74 - 81 .

BANERJEE S , LAVIE A . METEOR: An automatic metric for MT evaluation with improved correlation with human judgments [C]// Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization . Stroudsburg : Association for Computational Linguistics , 2005 : 65 - 72 .

KARAKANTA A , GAIDO M , NEGRI M , et al . Between flexibility and consistency: Joint generation of captions and subtitles [C]// Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021) . Stroudsburg : Association for Computational Linguistics , 2021 : 215 - 225 .

XU P , SAGHIR H , KANG J S , et al . A cross-domain transferable neural coherence model [C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2019 : 678 - 687 .

ELSNER M , CHARNIAK E . Coreference-inspired coherence modeling [C]// Proceedings of the Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2008 : 41 - 44 .

ROEMMELE M , GORDON A , SWANSON R . Evaluating story generation systems using automated linguistic analyses [C]// Proceedings of the SIGKDD 2017 Workshop on Machine Learning for Creativity . New York : ACM , 2017 : 13 - 17 .

GONG Z X , ZHANG M , ZHOU G D . Document-level machine translation evaluation with gist consistency and text cohesion [C]// Proceedings of the Second Workshop on Discourse in Machine Translation . Stroudsburg : Association for Computational Linguistics , 2015 : 33 - 40 .

MADOTTO A , LIN Z J , WU C S , et al . Personalizing dialogue agents via meta-learning [C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2019 : 5454 - 5459 .

WANG Z Y , WANG X Y , AN B , et al . Towards faithful neural table-to-text generation with content-matching constraints [C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : Association for Computational Linguistics , 2020 : 1072 - 1086 .

KANN K , ROTHE S , FILIPPOVA K . Sentence-level fluency evaluation: References help, but can be spared! [C]// Proceedings of the 22nd Conference on Computational Natural Language Learning . Stroudsburg : Association for Computational Linguistics , 2018 : 313 - 323 .

ZHANG T , KISHORE V , WU F , et al . Bertscore: Evaluating text generation with BERT [C]// Proceedings of the 8th International Conference on Learning Representations . Appleton : ICLR , 2020 : 1 - 43 .

LI W , WU W H , CHEN M Y , et al . Faithfulness in natural language generation: A systematic survey of analysis, evaluation and optimization methods [EB/OL]. ( 2022-03-10 )[ 2023-09-01 ]. http://arxiv.org/abs/2203.05227.pdf http://arxiv.org/abs/2203.05227.pdf .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于内容语义感知多模态融合的图像增强方法

面向知识图谱的二阶段复杂问句生成框架

融合引导注意力的中文长文本摘要生成

基于传播树的多特征谣言检测方法