CodeFuse-7b-4k fc_data 98.91 99.87 99.87 99.18 100 89.5 ⏬ Data Download Method 1: Download the zip file (you can also simply open the following link with the browser): wget https://huggingface.co/datasets/codefuse-admin/devopseval-exam/resolve/main/devopseval-exam.zip then unzi...
path.sep + eval_args.eval_language + os.path.sep + 'dev' + os.path.sep + dataset_fn_dict[dataset_name] df_dev = pd.read_csv(dev_dataset_fp) all_dataset[dataset_name] = preprocess(df, eval_args, df_dev=df_dev) logger.info('Load success, dataset_name={}, dataset_file_path...
Industrial-first evaluation benchmark for LLMs in the DevOps/AIOps domain. - 提交toollearning 评测代码与示例 · codefuse-ai/codefuse-devops-eval@63ac012
@@ -177,7 +177,8 @@ DevOps-Eval是一个专门为DevOps领域大模型设计的综合评估数据集 * 方法三:使用modelscope下载相关所有数据。示例如下: ```python from modelscope.msdatasets import MsDataset MsDataset.clone_meta(dataset_work_dir='./xxx', dataset_id='codefuse-ai/devopseval-exam')```...
DevOps-Eval is a comprehensive evaluation suite specifically designed for foundation models in the DevOps field. It consists of xxxx multi-choice questions spanning 8 diverse disciplines, as shown below. We hope DevOps-Eval could help developers, especially in the DevOps field, track the progres...
codefuse-devops-eval resources tool_learning_info.md onmain User selector All users DatepickerAll time Commit History Commits on Dec 27, 2023 Update README.md jimmy.xjcommittedDec 27, 2023 d703bae Update README.md jimmy.xjcommittedDec 27, 2023 c9cd51c Update README.md jimmy.xjcommit...
codefuse-ai / codefuse-devops-eval Public Notifications Fork 43 Star 681 Code Issues 5 Pull requests 1 Actions Projects 1 Security Insights CommitUpdate README.md Browse files main jimmy.xj committed Dec 27, 2023 1 parent e197e98 commit f53c4d8 ...
Industrial-first evaluation benchmark for LLMs in the DevOps/AIOps domain. - Merge pull request #19 from codefuse-ai/doc_update · codefuse-ai/codefuse-devops-eval@f0f12d4
codefuse-ai / codefuse-devops-eval Public Notifications Fork 43 Star 681 Code Issues 5 Pull requests 1 Actions Projects 1 Security Insights New issue 数据集需要进一步清洗 #2 Closed hhk123 opened this issue Nov 20, 2023· 1 comment ...
Industrial-first evaluation benchmark for LLMs in the DevOps/AIOps domain. - add funccall evalution features · codefuse-ai/codefuse-devops-eval@f7e2dff