Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilitiesArxiv: https://arxiv.org/pdf/2409.03444Github: https…
前言: prompting language model:将下游任务的输入形式构建成语言模型的输入那种能做一些下游任务。(将所有的下游任务的输入都转化成纯单个句子文本的形式,输出的内容也转化成语言模型对应的输出内容。)然而…
This paper introduces a selection-based LM using topic modeling for the purpose of domain adaptation which is often required in Statistical Machine Translation. The performance of this selection-based LM slightly outperforms the state-of-the- art Moore-Lewis LM by 1.0% for EN-ES and 0.7% for...
Language Model Adaptation for Slovak LVCSR Language model adaptation plays an important role in enhancing the performance of the automatic speech recognition systems, especially in case of domain-sp... Ján Stav,Daniel Hládek,Jozef Juhár 被引量: 15发表: 2010年 Integrating map, marginals, and uns...
Domain-specific adaptation is critical to maximizing the performance of pre-trained language models (PLMs) on one or multiple targeted tasks, especially under resource-constrained use cases, such as edge devices. However, existing methods often struggle to balance domain-specific performance, retention ...
In-context learning.The in-context learning (ICL) ability is formally introduced by GPT-3 [55]: assuming that the language model has been provided with a natural language instruction and/or several task demonstrations, it can generate the expected output for the test instances by completing the...
purely via text interaction with the model. GPT-3 achieves strong performance on many NLP datasets, including translation, question-answering, and cloze tasks, as well as several tasks that require on-the-fly reasoning or domain adaptation, such as unscrambling words, using a novel word in a ...
目前对于Domain Adaptation的研究方法主要可分为三类:Model-centric、Data-centric和融合前两者的Hybrid方法。在这篇文章中,作者主要关注了模型架构、特征空间增强、数据选择、预训练技术等问题,简单地融合了Model-centric和Data-centric方法,验证了有效进行域适应的一些策略。总体来讲本文虽然没有提出新颖的复杂模型,但是对...
In this paper we investigate random forest based language model adaptation. Large amounts of out-of-domain data are used to grow the decision trees while very small amounts of in-domain data are used to prune them back, so that the structure of the trees are suitable for the desired domai...
In this paper, we propose an unsupervised phrase-based data selection model, address the problem of selecting no-domain-specific language model (LM) training data to build adapted LM for use. In spoken language translation (SLT) system, we aim at finding the LM training sentences which are si...