pip install --upgrade"evalplus[perf,vllm] @ git+https://github.com/evalplus/evalplus"#Or `pip install "evalplus[perf,vllm]" --upgrade` for the latest stable releasesudo sh -c'echo 0 > /proc/sys/kernel/perf_event_paranoid'#Enable perfevalplus.evalperf --model"ise-uiuc/Magicoder-S...
C Repositories evalplusPublic Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024 Python1,382Apache-2.013445(2 issues need help)2UpdatedJan 6, 2025 evalplus.github.ioPublic repoqaPublic RepoQA: Evaluating Long-Context Code Understanding ...
Contribute to evalplus/evalperf_release development by creating an account on GitHub.
6 changes: 5 additions & 1 deletion 6 evalplus/evaluate.py Original file line numberDiff line numberDiff line change @@ -158,7 +158,11 @@ def evaluate( result_path = os.path.join(samples, "eval_results.json") else: assert samples.endswith(".jsonl") result_path = samples.replace(...
MBPP+:https://github.com/evalplus/mbppplus_release You can use EvalPlus throughbigcode-evaluation-harnessnow Docker image now uses Python 3.10 since some model might generate Python code using latest syntax, leading to false positive using older Python ...
git clone https://github.com/evalplus/repoqa.gitcdrepoqaexportPYTHONPATH=$PYTHONPATH:$(pwd)pip install -r requirements.txt 🏁 Search Needle Function (SNF) Search Needle Function is the first and base RepoQA task which aims to practice LLMs' ability oflong-context code understanding and ret...
.github codegen evalplus gallary tools _experimental humaneval mbpp tsr checker.py filter_inputs.py merge_dataset.py render.py sanitize.py stat_plus.py viz_passrate.py zip_solutions.py .dockerignore .gitignore .pre-commit-config.yaml CITATION.cff Dockerfile LICENSE MANIFEST.in README.md evo....
Release repository for MBPP+ data. Contribute to evalplus/mbppplus_release development by creating an account on GitHub.
51 51vllm = vllm>=0.5.152+gptqmodel = gptqmodel>=1.3.10 commit comments Comments0 (0)Footer © 2025 GitHub, Inc. Footer navigation Terms Privacy Security Status Docs Contact Manage cookies Do not share my personal information
Release repository for HumanEval+ data. Contribute to evalplus/humanevalplus_release development by creating an account on GitHub.