UpdatedJul 19, 2024 Python serialscriptr/Orange-PI-5-Pro-MLC-LLM Star9 Guide I wrote mostly for myself on how to run mlc-llm on the Orange Pi 5 Pro openclorange-pillmrk3588orange-pi-5mlc-llm UpdatedOct 10, 2024 Dockerfiles for poetry/mlc-llm(rk3588)/... ...
Code Folders and files Name Last commit message Last commit date Latest commit vicalloy add note to Makefile Sep 13, 2023 d4358de·Sep 13, 2023 History 43 Commits .github/workflows remove unused comment Sep 12, 2023 mlc-llm-rk3588
https://llm.mlc.ai/docs/install/gpu.html#orange-pi-5-rk3588-based-sbc https://blog.mlc.ai/2024/04/20/GPU-Accelerated-LLM-on-Orange-Pi https://github.com/Chrisz236/llm-rk3588 Please open an issue if you find something that can be improved in this guide See benchmarks for various ...
Building from source isn't working either (can't find tvm.so etc.). Could you please evaluate the build / install process on arm (RK3588 etc.) boards? There seem to be some recent issues. The original guide at https://blog.mlc.ai/2024/04/20/GPU-Accelerated-LLM-on-Orange-Pi doesn'...
what about mali610/rk3588? Seeing the Pascal GTX 10 series cards are supported then the P40 should work too I think. I have a couple of them and will test this out. asfarkTii mentioned this issue Oct 14, 2024 [Bug] Llama example inference using Vulkan gives build error #2977 Open ...