LMMs-Lab: Building Multimodal Intelligence We are a group of researchers, with a focus on large multimodal models (LMMs). We wish to bring insights to community with our research. Here're a few of our projects. LMMS-Eval We're on an exciting journey toward creating Artificial General Intelli...
【[25星]EvolvingLMMs-Lab/open-r1-multimodal:为 open-r1 添加多模态模型训练功能。亮点:1. 首批8k多模态RL训练样本,聚焦数学推理,数据丰富且质量高;2. 集成Qwen2-VL系列等VLMs,模型选择多样;3. 提供详细训练脚本和评估方法,易于上手和扩展】'Multimodal Open R1: A fork to add multimodal model training ...
Academics around the world use this theme for their homepages, blogs, lab pages, as well as webpages for courses, workshops, conferences, meetups, and more. Check out the community webpages below. Feel free to add your own page(s) by sending a PR....
多模态模型神经元解析新突破 | 在LLaVA等大型多模态模型中,神经元如何理解视觉与语言信号的融合一直是个黑箱。由EvolvingLMMs-Lab团队开发的Multimodal-SAE,首次通过稀疏自编码器(SAE)技术构建了可解释框架,其技术路径值得深入剖析: ▍技术路径解析:通过逆向工程分解神经元激活模式,可视化界面可实时观测不同输入刺激下的...
评估大模型多模态中的CoT推理能力 | 2025-02-09 ,由CUHK MMLab、CUHK MulLab、字节跳动、、东北大学等机构联合发布MME-CoT数据集,该数据集目的评估大型多模态模型(LMMs)中的思维链(CoT)推理能力,涵盖数学、科学、OCR、逻辑、时空和通用场景六个领域,通过细致的评估指标体系,深入分析了当前LMMs在推理质量、鲁棒性和...
If you would like to help with this ticket, we would love your help! Add yourself to the list of assignees on the ticket to the right,...
accelerate launch --num_processes=1 --main_process_port 12345 -m lmms_eval \ --model llava_onevision \ --model_args pretrained=lmms-lab/llava-onevision-qwen2-7b-ov,conv_template=qwen_1_5,model_name=llava_qwen,max_frames_num=1,torch_dype=bfloat16 \ --tasks video_mmmu_adaptation_...
{port_in_cmd}" \ # 12345 src/open_r1/grpo.py \ --deepspeed scripts/zero3.json \ --output_dir checkpoints/Qwen2-VL-2B-GRPO-8k \ --model_name_or_path Qwen/Qwen2-VL-2B-Instruct \ --dataset_name lmms-lab/multimodal-open-r1-8k-verified \ --max_prompt_length 8192 \ --per_...
EvolvingLMMs-Lab / VideoMMMU Public Notifications Fork 1 Star 23 Code Issues 1 Pull requests 1 Actions Projects Security Insights Preview Issues Search Issues is:issue state:open LabelsMilestonesNew issueFooter © 2025 GitHub, Inc. Footer navigation Terms Privacy Security Status Docs ...
I want to understand on the use of Model Architecture difference between the author release oflmms-laband the HF team releases onllava-hf. For the same set of Models does using one over the another has performance difference? And is there any plans to transfer weights trained on one to ano...