async for generation in llm.generate_iterator('What is the meaning of life?'): print(generation.outputs[0].text)Note The motivation behind making llm.generate_iterator an async generator is to provide support for Continuous batching with vLLM backend. By having the async endpoints, each prompt...
act out an illness. then show the other new words on the blackboard. read the new words to students and ask them to repeat. make sure every student knows the meaning and can read them. step iv pairwork teach students more words of illness. for example, have a cold, have a fever, h...
async for generation in llm.generate_iterator('What is the meaning of life?'): print(generation.outputs[0].text)Note The motivation behind making llm.generate_iterator an async generator is to provide support for Continuous batching with vLLM backend. By having the async endpoints, each prompt...