Run Open Source/Open Weight LLMs locally with OpenAI compatible APIs Resources Readme Activity Custom properties Stars 92 stars Watchers 5 watching Forks 6 forks Report repository Releases 24 Bodhi v0.0.19 Latest Feb 5, 2025 + 23 releases Packages No packages published Contributors ...
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance. - Support OpenAI-Compatible APIs (#123) · lihuibng/lightllm@4f42716
Build docsearch with a crawlerConnecting to other APIs MongoDB: Vector and geospatial search over satellite imagesDiscord: Deploy and run a Discord BotGoogle Sheets: Sync databases and APIs to a Google SheetLangChain: Run a RAG Q&A chatbotTailscale: Add Modal Apps to your VPNPrometheus: Publish...
NVIDIA GenAI-Perf currently supports three OpenAI endpoint APIs:Chat Chat Completions EmbeddingsNew endpoints will be released as new model types become popular. GenAI-Perf is also open source and accepts community contributions.Running GenAI-Perf...
如何使用FastChat? | github: https://github.com/lm-sys/FastChatFastChat's core features include:The training and evaluation code for state-of-the-art models (e.g., Vicuna, MT-Bench).A distributed multi-model serving system with web UI and OpenAI-compatible RESTful APIs. ...
Sometimes, you might have applications developed using OpenAI APIs or SDKs, and you want to experiment with Amazon Bedrock without modifying your codebase. Or you may simply wish to evaluate the capabilities of these foundation models in tools like AutoGen etc. Well, this repository allows you ...
Add an option to forward user information to the external OpenAI-compatible API URL, either as part of the payload or in the headers. This feature would be beneficial when integrating with OpenAI-compatible APIs that can make use of user-specific data. ...
Configuring Kokoro Fast API within the same Open WebUI instance and testing with the play button worked successfully, indicating the TTS functionality is operational but fails specifically with custom OpenAI-compatible APIs. Possible Cause: The issue may be related to how Open WebUI handles custom ...
Feature request prepend v1 to OpenAI compatible APIs Motivation This allows us to integrate infinity the same way as other openai compatible API engines into KubeAI: https://github.com/substratusai/kubeai PR: substratusai/kubeai#197 Your...
As assistant API is still in Beta and is super slow, so we don't have plan to support it (and relevant file APIs) for now. Examples Here are some examples of how to use the SDK: // create image let sdk = LlmSdk::new("https://api.openai.com/v1", "your-api-key"); let req...