[NAACL'24] Self-data filtering of LLM instruction-tuning data using a novel perplexity-based difficulty score, without using any other models - ura-hcmut/Cherry_LLM
score.py tidy Feb 10, 2025 Very simple example of a Perplexity calculation based on wikipedia trained model. Based on filtering the common crawl from:https://github.com/facebookresearch/cc_net& accompanying paper:https://arxiv.org/pdf/1911.00359 ...
The IFD score proposed by us can divide the samples into better or relatively bad ones, which might provide insight into the types of data good for instruction tuning. (Selective Reflection-Tuning) The IFD scores and the reversed version can be utilized to construct better data! In Reflection-...
The IFD score proposed by us can divide the samples into better or relatively bad ones, which might provide insight into the types of data good for instruction tuning. (Selective Reflection-Tuning)The IFD scores and the reversed version can be utilized to construct better data! InReflection-Tuni...
The IFD score proposed by us can divide the samples into better or relatively bad ones, which might provide insight into the types of data good for instruction tuning. (Selective Reflection-Tuning) The IFD scores and the reversed version can be utilized to construct better data! In Reflection-...
[NAACL'24] Self-data filtering of LLM instruction-tuning data using a novel perplexity-based difficulty score, without using any other models - tianyi-lab/Cherry_LLM