The secret is to use openai JSON style of output in your local LLM server such as Oobabooga’s text-generation-webui, then hook it to autogen. That’s what we’re building today. Note there are other methods for making llms spit text in openai apis format as well like the llama.cpp...
When using APIhttp://0.0.0.0:5000how to continue the chat session before? Any idea? TiagoTiagocommentedMay 31, 2024 I think in that case, keeping up history would depend on the client app used; not sure OTGWUI records anything that comes thru that way. I could be wrong though. ...
I feel i make some mistake and now I want to uninstall it completely. How to do that? Contributor Brawlence commented May 5, 2023 Hmmm. Complete uninstallation would include: removing the text-gen-web-UI folder removing the venv folder (probably) removing torch hub local cache dir in you...
Under WIndows and without WSL I have successfully compiled deepspeed (deepspeed-0.9.3+unknown-cp310-cp310-win_amd64.whl). So how to install this specific file it in the current conda environment to be able to use the command "deepspeed -...