[rank0]: return model_class(vllm_config=vllm_config, prefix=prefix) [rank0]: ^^^ [rank0]: File "/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/deepseek_v3.py", line 505, ininit [rank0]: self.model = DeepseekV3Model(vllm_config=vllm_config, [rank0]: ^^...
tokenizer='models/llama-2-7b-hf', tokenizer_mode=auto, trust_remote_code=False, dtype=torch.float16, use_dummy_weights=False, download_dir=None, use_np_weights=False, tensor_parallel_size=2, seed=0)
Is debug build: False CUDA used to build PyTorch: 12.1 ROCM used to build PyTorch: N/A OS: Ubuntu 22.04.4 LTS (x86_64) GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 Clang version: Could not collect CMake version: version 3.29.5 Libc version: glibc-2.35 Python version: 3.11....
tensor_parallel_size=2, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, quantization_param_path=None, device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='outlines'), seed=0, served_model_name=/scratch/pbs.5401450.kman.restech....
[rank0]: File "/mnt/data/Pai-Megatron-Patch/PAI-Megatron-LM-240718/megatron/core/pipeline_parallel/schedules.py", line 1344, in forward_backward_pipelining_without_interleaving [rank0]: output_tensor, num_tokens = forward_step( [rank0]: File "/mnt/data/Pai-Megatron-Patch/PAI-Megatron-LM-...
Pull requests467 Discussions Actions Projects4 Security2 Insights Additional navigation options Description huangyunxin whyiug commentedon Feb 1, 2024 whyiug lzhfe commentedon Feb 2, 2024 lzhfe skyantao commentedon Feb 7, 2024 skyantao hediyuan commentedon Feb 20, 2024 ...