Tailored-LLaMA: Optimizing Few-Shot Learning in Pruned LLaMA Models with Task-Specific Prompts Danyal Aftab, Steven Davy Paper LLMCBench: Benchmarking Large Language Model Compression for Efficient Deployment Ge Yang, Changyi He, Jinyang Guo, Jianyu Wu, Yifu Ding, Aishan Liu, Haotong Qin, Peng...
MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Dataset. 2024. [arxiv] Hu et al. Computational Limits of Low-Rank Adaptation (LoRA) for Transformer-Based Models. 2024. [arxiv] Ge et al. Time Sensitive Knowledge Editing through Efficient ...
Evaluation and Benchmarking of Efficient Models 📊 Efficient Solutions in Other Modalities and Applications 🌐 Efficiency of foundational or pre-trained models in multi-modal set-up and other modalities (beyond NLP and Speech) such as biology, chemistry, computer vision, and time series Efficient ...
a, Average runtime of different methods for generating 100 protein pockets for a ligand molecule on the two benchmarks. Data are presented as mean ± standard deviation. The sample size for each method is 100.b, Trade-off between quality (measured by Vina score) and diversity (1 –...
In this paper, we propose a parameter-efficient fine-tuning framework that effectively increases MRC capabilities on decoder-only large language models. This framework designs the process for MRC and introduces the low-rank adaptation (LoRA) method to effectively fine-tune the large model with many...
Natural evolution must explore a vast landscape of possible sequences for desirable yet rare mutations, suggesting that learning from natural evolutionary strategies could guide artificial evolution. Here we report that general protein language models can efficiently evolve human antibodies by suggesting muta...
Rather than extracting slots through weak-supervision or transfer learning, the method of [44] extracts slots completely unsupervised by using self-supervised language models trained on the task-specific dialogues and unsupervised parsers to identify slot candidates, after which these are similarly ...
(Contextual Understanding) Video-based Generative Performance Benchmarking (Correctness of Information) Video-based Generative Performance Benchmarking (Detail Orientation)) Video-based Generative Performance Benchmarking (Temporal Understanding) Video Question Answering Visual Question Answering Visual Question ...
MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Dataset. 2024. [arxiv] Hu et al. Computational Limits of Low-Rank Adaptation (LoRA) for Transformer-Based Models. 2024. [arxiv] Ge et al. Time Sensitive Knowledge Editing through Efficient ...
--check Whether to compute perplexity during benchmarking for verification. --cuda CUDA GPU device string, 'cuda:0' by default. --eval Evaluate the model with dataset wikitext2, ptb and c4 Quantize 7B model to 8-bit python -m llama.llama_quant decapoda-research/llama-7b-hf c4 --wbits...