apiKey: null, }); this.model = modelPreference ?? process.env.KOBOLD_CPP_MODEL_PREF ?? null; if (!this.model) throw new Error("KoboldCPP must have a valid model set."); this.limits = { history: this.promptWindowLimit() * 0.15, system: this.promptWindowLimit() * 0.15, user: th...
It's a single self contained distributable from Concedo, that builds off llama.cpp, and adds a versatile Kobold API endpoint, additional format support, Stable Diffusion image generation, backward compatibility, as well as a fancy UI with persistent stories, editing tools, save formats, memory, ...
VE FORBRYDERNE - Contributed many features such as the Editing overhaul, Adventure Mode, expansions to the world info section, breakmodel integration, scripting support, API, softpromtps and much more. As well as vastly improving the TPU compatibility and integrating external code into KoboldAI so...
git config --global user.name userName git config --global user.email userEmail koboldcpp / ggml-backend.h ggml-backend.h12.92 KB 一键复制编辑原始数据按行查看历史 slaren提交于11个月前.llama : add pipeline parallelism support (#6017) ...
NEW: Experimental ComfyUI Support Added!: ComfyUI can now be used as an image generation backend API from within KoboldAI Lite. No workflow customization is necessary.Note: ComfyUI must be launched with the flags --listen --enable-cors-header '*' to enable API access.Then you may use it...
VE FORBRYDERNE - Contributed many features such as the Editing overhaul, Adventure Mode, expansions to the world info section, breakmodel integration, scripting support, API, softpromtps and much more. As well as vastly improving the TPU compatibility and integrating external code into KoboldAI so...
The api_like_OAI.py script has been removed - use server instead (#5766) Support for chat templates: Wiki (contributions welcome) Support for Gemma models: ggerganov#5631 Non-linear quantization IQ4_NL: ggerganov#5590 Looking for contributions to improve and maintain the server example: gge...
Premium Support Enterprise-grade 24/7 support Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Provide feedback We read every piece of feedback, and take your input very seriously. Include my email address so I can be contacted Cancel Submit ...
Premium Support Enterprise-grade 24/7 support Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Provide feedback We read every piece of feedback, and take your input very seriously. Include my email address so I can be contacted Cancel Submit feed...
For Jetson user, if you have Jetson Orin, you can try this: Offical Support. If you are using an old model(nano/TX2), need some additional operations before compiling. Using make: make LLAMA_CUDA=1 Using CMake: cmake -B build -DLLAMA_CUDA=ON cmake --build build --config Release ...