robust fine-tuning of zero-shot models "Robust fine-tuning of zero-shot models"是指对零样本模型进行稳健的微调。在机器学习中,零样本学习是指模型在没有见过特定任务的数据情况下,能够对该任务进行推断或预测。 在零样本学习中,通常使用预训练的模型,然后在新任务上进行微调,以适应特定的任务。然而,由于新...
论文的核心内容是提出了一种名为WiSE-FT(Weight-space ensembling for Fine-tuning)的方法,用于在保持零样本(zero-shot)模型的鲁棒性的同时,对其进行微调(fine-tuning)以提高在特定目标分布上的准确性。零样本模型,如CLIP或ALIGN,在没有针对特定数据集进行微调的情况下,能够在一系列数据分布上保持一致的准确性。然而...
Robust fine-tuning of zero-shot models Mitchell Wortsman∗ University of Washington mitchnw@cs.washington.edu Gabriel Ilharco∗ University of Washington gamaga@cs.washington.edu Jong Wook Kim OpenAI jongwook@openai.com Mike Li Columbia University mli24@gsb.columbia.edu Simon Kornblith Google ...
Large pre-trained models such as CLIP or ALIGN offer consistent accuracy across a range of data distributions when performing zero-shot inference (i.e., without fine-tuning on a specific dataset). Although existing fine-tuning methods substantially improve accuracy on a given target distribution, ...
Fine-tuningRobustnessContrastive language-image pre-trained (CLIP) models have zero-shot ability of classifying an image belonging to "[CLASS]" by using similarity between the image and the prompt sentence "a [CONTEXT] of [CLASS]". Based on exhaustive text cues in "[CONTEXT]", CLIP model ...
Geodesic Multi-Modal Mixup for Robust Fine-Tuning Pre-trained multi-modal models, such as CLIP, provide transferable embeddings and show promising results in diverse applications. However, the analysis of learned multi-modal embeddings is relatively unexplored, and the embedding transferability can be ...
distributionally robust finetuning bert for covariate drift in ... [Paper] adversarial adaptation of synthetic or stale data [Paper] semi-supervised domain adaptation for dependency parsing ... [Paper] joint and conditional estimation of tagging and parsing models [Paper] measure and improve ...
* Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models * 链接:https://arxiv.org/abs/2209.07511 * 作者: Manli Shu,Weili Nie,De-An Huang,Zhiding Yu,Tom Goldstein,Anima Anandkumar,Chaowei Xiao * 其他: NeurIPS 2022 ...
We demonstrate models trained at this scale transfer well to existing datasets zero- shot, removing the need for any dataset-specific fine-tuning to achieve high-quality results. In addition to scale, our work also focuses on broaden- ing the scope of weakly supervised pre-training beyond ...
few-shot prompt tuning fixed classname tokens zero-shot learning downstream tasks: few-shot learning, continual learning, object segmentation model-informed structure traditional fine-tuning and linear probing paradigms generalized cross-entropy (GCE) VisionLanguage Pre-Trained Models (VL-PTMs) meta-learn...