54 54 如果部署环境是Linux,推荐安装fast_tokenizer可以得到更极致的文本处理效率,进一步提升服务性能。目前暂不支持Windows设备安装,将会在下个版本支持。 55 55 ```shell 56 - pip install fast_tokenizer 56 + pip install fast-tokenizer-python 57 57 ``` --> 58 58 59 59 Diff for: fast...
🚀 Feature request Tokenizer are provided with each model, some have a fast version of their tokenizer (Rust based), others like CamemBERT have only the slow version. Motivation Fast tokenizer improves inference times drastically (in real...
目前由于人力问题，已经停止更新fast-tokenizer了，欢迎开发者贡献。https://github.com/PaddlePaddle/Paddle...