1.注册c-eval, 一个适用于大语言模型的多层次多学科中文评估套件2.下载ceval数据集 wget https://huggingface.co/datasets/ceval/ceval-exam/resolve/main/ceval-exam.zip3.跑出模型在ceval的test数据集上的预测值…
具体的数据在HuggingFace官网中Datasets下搜索ceval-exam既可进行预览,选择Subset为college_programming,数据分割为dev,预览5条大学编程试题。 C-Eval的HuggingFace数据预览 C-Eval的Prompt范式 分别有两种Prompt提示语方式来引导模型给出答案,一种是answer-only,一种是chain-of-thought,answer-only指的是不是用思维链,直...
数据下载: wgethttps://huggingface.co/datasets/ceval/ceval-exam/resolve/main/ceval-exam.zip /root/ChatGLM-6B/ptuning/ceval/data/val/computer_architecture_val.csv data/val/computer_architecture_val.csv /root/ChatGLM-6B/ptuning/ceval/data/dev/computer_architecture_dev.csv Isaac 张雯轩:LLM Eval...
Method 1: Download the zip file (you can also simply open the following link with the browser): wget https://huggingface.co/datasets/ceval/ceval-exam/resolve/main/ceval-exam.zip then unzip it and you may load the data with pandas: ...
Method 1: Download the zip file (you can also simply open the following link with the browser): wget https://huggingface.co/datasets/ceval/ceval-exam/resolve/main/ceval-exam.zip then unzip it and you may load the data with pandas: import os import pandas as pd File_Dir="ceval-exam...
具体的数据在HuggingFace官网中Datasets下搜索ceval-exam既可进行预览,选择Subset为college_programming,数据分割为dev,预览5条大学编程试题。 C-Eval的HuggingFace数据预览 C-Eval的Prompt范式 分别有两种Prompt提示语方式来引导模型给出答案,一种是answer-only,一种是chain-of-thought,answer-only指的是不是用思维链,直...
wget https://huggingface.co/datasets/ceval/ceval-exam/resolve/main/ceval-exam.zip mkdir data/ceval mv ceval-exam.zip data/ceval cd data/ceval; unzip ceval-exam.zip cd ../../ pip install thefuzz python eval/evaluate_chat_ceval.py -d data/ceval ''' def load_models...
wget https://huggingface.co/datasets/ceval/ceval-exam/resolve/main/ceval-exam.zip mkdir data/ceval mv ceval-exam.zip data/ceval cd data/ceval; unzip ceval-exam.zip cd ../../ # Qwen-7B python evaluate_ceval.py -d data/ceval/ # Qwen-7B-Chat pip install thefuzz python evaluate...
Search or jump to... Search code, repositories, users, issues, pull requests... Provide feedback We read every piece of feedback, and take your input very seriously. Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your...
class CevalExam(datasets.GeneratorBasedBuilder): BUILDER_CONFIGS = [ CevalExamConfig( name=task_name, ) for task_name in task_list ] def _info(self): features = datasets.Features( { "id": datasets.Value("int32"), "question": datasets.Value("string"), "A": datasets.Value("string")...