Security Insights Additional navigation options This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. Browse files jasonkylelol committedDec 20, 2024 fix: tiktoken cl100k_base offline download ...
Below are the file download links:p50k_base.tiktokencl100k_base.tiktokeno200k_base.tiktoken Benchmark Test I noticed that some users would like to get a comparison of efficiency. Here, I use SharpToken as the basic comparison, with the encoder cl100k_base, on the .Net 6.0 in Debug mode...