Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. INFO 05-30 16:17:09 utils.py:638] Found nccl from library /home1/zxj/.config/vllm/nccl/cu12/libnccl.so.2.18.1 (RayWorkerWrapper pid=93372) INFO 05-30 16:17:09 uti...
Downloading vllm_nccl_cu12-2.18.1.0.4.0.tar.gz (6.2 kB) Preparingmetadata(setup.py) ... done 此处略去 ... Successfully installed diskcache-5.6.3 dnspython-2.6.1 email_validator-2.1.1 fastapi-0.111.0 fastapi-cli-0.0.3 h11-0.14.0 httpcore-1.0.5 httptools-0.6.1 httpx-0.27.0 interegu...
nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-...
Pull requests Actions Projects Security Insights Additional navigation options Files main vllm_nccl .gitignore LICENSE README.md setup.py Latest commit Cannot retrieve latest commit at this time. History History Breadcrumbs vllm-nccl / File metadata and controls ...
A high-throughput and memory-efficient inference and serving engine for LLMs - vllm/csrc/custom_all_reduce_test.cu at main · isuyu/vllm
同样的错误也发生在我身上。这个bug还在持续吗？
Solutions By company size Enterprises Small and medium teams Startups By use case DevSecOps DevOps CI/CD View all use cases By industry Healthcare Financial services Manufacturing Government View all industries View all solutions Resources Topics AI DevOps Security Software Development...
We read every piece of feedback, and take your input very seriously. Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Cancel Create saved search Sign in Sign up Reseting focus {...
Search or jump to... Search code, repositories, users, issues, pull requests... Provide feedback We read every piece of feedback, and take your input very seriously. Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter ...
[pip3] vllm-nccl-cu12==2.18.1.0.3.0 [conda] Could not collectROCM Version: Could not collect Neuron SDK Version: N/A vLLM Version: 0.4.1 vLLM Build Flags: CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled GPU Topology: