RWKV infctx trainer, for training arbitary context sizes, to 10k and beyond! - RWKV-infctx-trainer/notebook/dataset-config at main · xiaol/RWKV-infctx-trainer
https://github.com/RWKV/RWKV-infctx-trainerInfctx trainer https://github.com/Blealtan/RWKV-LM-LoRALoRA finetuning https://github.com/TheRamU/Fay/blob/main/README_EN.mdDigital Assistant with RWKV https://github.com/harrisonvanderbyl/rwkv-cpp-cudaFast GPU inference with cuda/amd/vulkan RW...
https://github.com/RWKV/RWKV-infctx-trainer Infctx trainer https://github.com/Blealtan/RWKV-LM-LoRA LoRA finetuning https://github.com/TheRamU/Fay/blob/main/README_EN.md Digital Assistant with RWKV https://github.com/harrisonvanderbyl/rwkv-cpp-cuda Fast GPU inference with cuda/amd/vul...
https://github.com/RWKV/RWKV-infctx-trainerInfctx trainer https://github.com/Blealtan/RWKV-LM-LoRALoRA finetuning World demo script:https://github.com/BlinkDL/ChatRWKV/blob/main/API_DEMO_WORLD.py Raven Q&A demo script:https://github.com/BlinkDL/ChatRWKV/blob/main/v2/benchmark_more....
https://github.com/RWKV/RWKV-infctx-trainerInfctx trainer https://github.com/daquexian/faster-rwkv https://github.com/mlc-ai/mlc-llm/pull/1275 https://github.com/TheRamU/Fay/blob/main/README_EN.mdDigital Assistant with RWKV
最后是另一个RWKV无限上下文训练的项目,如果你有更高的训练/微调需求,可以使用此项目:https://github.com/RWKV/RWKV-infctx-trainer 你也可以在这个网址查看一些相关说明:https://github.com/SynthiaDL/TrainChatGalRWKV 由于这个项目有比较详细的介绍,这里就不多赘述了,此外RWKV-Runner已经自动进行了环境部署,你...
https://github.com/RWKV/RWKV-infctx-trainerInfctx trainer World demo script:https://github.com/BlinkDL/ChatRWKV/blob/main/API_DEMO_WORLD.py Raven Q&A demo script:https://github.com/BlinkDL/ChatRWKV/blob/main/v2/benchmark_more.py ...
trainer.my_wandb = wandb def on_train_batch_end(self, trainer, pl_module, outputs, batch, batch_idx): args = self.args token_per_step = args.ctx_len * args.real_bsz real_step = trainer.global_step + args.epoch_begin * args.epoch_steps if trainer.is_global_zero: # logging t_now...
最后是另一个RWKV无限上下文训练的项目,如果你有更高的训练/微调需求,可以使用此项目:https://github.com/RWKV/RWKV-infctx-trainer 你也可以在这个网址查看一些相关说明:https://github.com/SynthiaDL/TrainChatGalRWKV 由于这个项目有比较详细的介绍,这里就不多赘述了,此外RWKV-Runner已经自动进行了环境部署,你...