目前由于人力问题,已经停止更新fast-tokenizer了,欢迎开发者贡献。https://github.com/PaddlePaddle/PaddleNLP/tree/develop/fast_tokenizer
🚀 Feature request Tokenizer are provided with each model, some have a fast version of their tokenizer (Rust based), others like CamemBERT have only the slow version. Motivation Fast tokenizer improves inference times drastically (in real...