56 - pip install fast_tokenizer 56 + pip install fast-tokenizer-python 57 57 ``` --> 58 58 59 59 Diff for: fast_tokenizer/setup.py +2-1 Original file line numberDiff line numberDiff line change @@ -64,7 +64,7 @@ def get_version(): 64 64 65 65 long_description...
目前由于人力问题，已经停止更新fast-tokenizer了，欢迎开发者贡献。https://github.com/PaddlePaddle/Paddle...
🚀 Feature request Tokenizer are provided with each model, some have a fast version of their tokenizer (Rust based), others like CamemBERT have only the slow version. Motivation Fast tokenizer improves inference times drastically (in real...