To use GPT-J, download the HuggingFace-compatible model checkpoint provided by EleutherAI. Evaluation For supported models, answers and scores can be generated by running truthfulqa/evaluate.py with the appropriate flags. To test the performance of a new model on the generation task, add its ans...
--cache_dirLocation of cached HuggingFace models --gptj_pathLocation of GPT-J checkpoint Model classModels GPT-3ada,babbage,curie,davinci GPT-Neo/Jneo-small,neo-med,neo-large,gptj GPT-2gpt2,gpt2-xl UnifiedQAuqa-small,uqa-base,uqa-large,uqa-3b ...