[Bugfix] Fix prompt format of GLM4V (vllm-project#14539) … 43d1a79 richardsliu pushed a commit to richardsliu/vllm that referenced this pull request Mar 14, 2025 [Bugfix] Fix prompt format of GLM4V (vllm-project#14539) … 4a5cebd Sign up for free to join this conversation ...
inputs = tokenizer.apply_chat_template([{"role": "user", "content": query}], add_generation_prompt=True, tokenize=True, return_tensors="pt", return_dict=True) Expected behavior / 期待表现 想确认一下 GLM4V是否支持纯文本微调 如果支持纯文本微调,是否同样是使用finetune_vision.py 纯文本微调的...
{"role": "user", "image": image, "content": query}], add_generation_prompt=True, tokenize=True, return_tensors="pt", return_dict=True) # chat mode inputs = inputs.to(device) model = AutoModelForCausalLM.from_pretrained( "THUDM/glm-4v-9b", torch_dtype=torch.bfloat16, low_cpu...
File "/root/anaconda3/envs/glm4v-9b-vLLM0_6_3_post1/lib/python3.11/site-packages/vllm/inputs/parse.py", line 98, in parse_singleton_prompt raise TypeError("inputs must be a string, TextPrompt, or TokensPrompt") TypeError: inputs must be a string, TextPrompt, or TokensPrompt The a...
@@ -93,7 +93,7 @@ def construct_prompt(self, question: str) -> str: #Currently, does not work! class DeepseekVL2(BaseLLM): def __init__(self, model_name: str = "deepseek-ai/deepseek-vl2-tiny", **kwargs): super().__init__(model_name, **kwargs)...
") E RuntimeError: Expected there to be 1 prompt updates corresponding to 1 image items, but instead found 0 prompt updates! Either the prompt text has missing/incorrect tokens for multi-modal inputs, or there is a problem with your implementation of merged multi-modal processor for this ...
@@ -115,10 +115,11 @@ def construct_prompt(self, question: str) -> str:class GLM4V(BaseLLM): def __init__(self, model_name: str = "THUDM/glm-4v-9b", **kwargs): super().__init__(model_name) super().__init__(model_name, **kwargs)...
prompt_tokens: int = 0 total_tokens: int = 0 completion_tokens: Optional[int] = 0 class ChatCompletionResponse(BaseModel): model: str object: Literal["chat.completion", "chat.completion.chunk"] choices: List[Union[ChatCompletionResponseChoice, ChatCompletionResponseStreamChoice]] created: Opt...
Vision Prompt Tuning: Visual Prompt Tuning Side: Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks Res-Tuning: Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbone < arXiv \ Tuners provided by PEFT, such as IA3, AdaLoRA, etc. Supporte...
Vision Prompt Tuning: Visual Prompt Tuning Side: Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks Res-Tuning: Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbone < arXiv \ Tuners provided by PEFT, such as IA3, AdaLoRA, etc. Supporte...