[2024/05/10] 🔥LLaVA-NeXT(Stronger) models are released, stronger LMM with support of LLama-3 (8B) and Qwen-1.5 (72B/110B). [Blog] [Checkpoints] [Demo] [Code] [2024/05/10] 🔥LLaVA-NeXT(Video) is released. The
扩大LLM提供更强的语义理解和推理能力,支持llama3(8B),Qwen(72B和110B)等模型 扩充评估数据集 LLaVA-NeXT(Video) 详情链接:LLaVA-NeXT: A Strong Zero-shot Video Understanding Model 更新说明 扩展LLava-NeXT到视频理解任务:采用AnyRes技术很轻松的将图像任务扩展到视频任务(视频相当于扩充了token长度) Length ge...
[2024/05/10] 🔥LLaVA-NeXT(Stronger) models are released, stronger LMM with support of LLama-3 (8B) and Qwen-1.5 (72B/110B). [Blog] [Checkpoints] [Demo] [Code] [2024/05/10] 🔥LLaVA-NeXT(Video) is released. The image-only-trained LLaVA-NeXT model is surprisingly strong on vid...
LLM 和 visual encoder 都是预训练好的自回归结构和 ViT 模型,各个多模态的模型在结构上的区别一般都在 adapter 这里,常见的有 linear、cross attention、q-former、perceiver resampler 等。 LLaVA 这里使用的 adaper 就是最简单的 linear 线性映射,采用 CLIP 的ViT-L/14作为 visual encoder,采用Vicuna(LLaMa的...
# Single GPU CUDA_VISIBLE_DEVICES=0 python3 -m sglang.launch_server --model-path liuhaotian/llava-v1.5-7b --tokenizer-path llava-hf/llava-1.5-7b-hf --port 30000 # Multiple GPUs with tensor parallel CUDA_VISIBLE_DEVICES=0,1 python3 -m sglang.launch_server --model-path liuhaotian/ll...
liuhaotian/llava-v1.5-7b --tokenizer-path llava-hf/llava-1.5-7b-hf --port 30000 # Multiple GPUs with tensor parallel CUDA_VISIBLE_DEVICES=0,1 python3 -m sglang.launch_server --model-path liuhaotian/llava-v1.5-13b --tokenizer-path llava-hf/llava-1.5-13b-hf --port 30000 --tp 2...
liuhaotian/llava-v1.5-7b --tokenizer-path llava-hf/llava-1.5-7b-hf --port 30000 # Multiple GPUs with tensor parallel CUDA_VISIBLE_DEVICES=0,1 python3 -m sglang.launch_server --model-path liuhaotian/llava-v1.5-13b --tokenizer-path llava-hf/llava-1.5-13b-hf --port 30000 --tp 2...
liuhaotian/llava-v1.5-7b --tokenizer-path llava-hf/llava-1.5-7b-hf --port 30000 # Multiple GPUs with tensor parallel CUDA_VISIBLE_DEVICES=0,1 python3 -m sglang.launch_server --model-path liuhaotian/llava-v1.5-13b --tokenizer-path llava-hf/llava-1.5-13b-hf --port 30000 --tp 2...
# Single GPU CUDA_VISIBLE_DEVICES=0 python3 -m sglang.launch_server --model-path liuhaotian/llava-v1.5-7b --tokenizer-path llava-hf/llava-1.5-7b-hf --port 30000 # Multiple GPUs with tensor parallel CUDA_VISIBLE_DEVICES=0,1 python3 -m sglang.launch_server --model-path liuhaotian/ll...
liuhaotian/llava-v1.5-7b --tokenizer-path llava-hf/llava-1.5-7b-hf --port 30000 # Multiple GPUs with tensor parallel CUDA_VISIBLE_DEVICES=0,1 python3 -m sglang.launch_server --model-path liuhaotian/llava-v1.5-13b --tokenizer-path llava-hf/llava-1.5-13b-hf --port 30000 --tp 2...