Skip to content

Issues: open-compass/opencompass

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

273 Open 372 Closed
273 Open 372 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Bug] VLLM推理时参数不匹配
#1935 opened Mar 12, 2025 by c-box
2 tasks done
@MaiziXiao
[Bug] mmlu_pro结果正则提取出错
#1933 opened Mar 11, 2025 by wangzhaode
2 tasks done
@liushz
4
[Bug] LiveCodeBench评测报错
#1930 opened Mar 10, 2025 by AllenShow
2 tasks done
[Bug] math_500_gen评测报错
#1929 opened Mar 10, 2025 by AllenShow
2 tasks done
[Bug] gpqa_gen数据集得出的结果很低
#1928 opened Mar 10, 2025 by AllenShow
2 tasks done
2
[Bug] 使用deepseek-v2-lite-chat进行humaneval测试时，出现问题，无论是在官网还是本地部署测试分数只有1.22，其他模型都正常
#1926 opened Mar 10, 2025 by shuowoshishui
2 tasks done
@Zhudongsheng75
3
[Feature] Cannot find a huggingafce source code to reload model找不到加载hf模型的源代码
#1921 opened Mar 7, 2025 by msz12345
1 task
@tonysy
[Bug] Please help.Unable to evaluate Deepseek-R1-Qwen-32B 4-bit quantized version
#1918 opened Mar 5, 2025 by t822876884
2 tasks done
4
[Feature] Offering help with Unit Tests
#1915 opened Mar 5, 2025 by erlapso
@MaiziXiao
[Bug] Deepseek-R1-Distill-32B 评测报错
#1914 opened Mar 5, 2025 by t822876884
2 tasks done
@MaiziXiao
5
[Bug] xunfei api 评测报错
#1913 opened Mar 5, 2025 by wtlwang2019
2 tasks done
1
[Feature] 是否有计划兼容python3.11
#1911 opened Mar 4, 2025 by linbeyoung
1 task
@MaiziXiao @liushz
1
[Bug] no error during the evaluation runtime，but the result was empty, and it said "Floating point exception (core dumped)" in output
#1903 opened Mar 3, 2025 by azuercici
2 tasks done
@MaiziXiao
3
[Bug] Take too much time on MATH-500 dataset evaluation
#1895 opened Feb 26, 2025 by msz12345
2 tasks done
@MaiziXiao
4
[Bug] 使用max-num-worker使得ssh断连
#1887 opened Feb 23, 2025 by timturing
2 tasks done
@tonysy
3
[Feature] Can we support terminate dlc and volc tasks when oc evaluation task is terminated.
#1884 opened Feb 20, 2025 by zhulinJulia24
1 task
@acylam
[Bug] human-eval assertion failed while testing partial data
#1880 opened Feb 19, 2025 by dengbinbox
2 tasks done
[Bug] DeepSeek R1 32B 模型 测评 AIME2024 数据集 得分低
#1878 opened Feb 18, 2025 by carllisicau
2 tasks done
11
[Feature] 我需要对本地部署的Qwen-110B模型进行MMLU基准测试，请问该怎么操作呢？
#1877 opened Feb 18, 2025 by hi112233445566
1 task
[Feature] 目前是否有适配Codeforces、SWE Verified、Aider-Polyglot这些在R1中出现的数据集的计划呢？
#1875 opened Feb 17, 2025 by linbeyoung
1 task
@bittersweet1999
2
[Bug] Medbench dataset only provides test data, not the entire dataset
#1874 opened Feb 16, 2025 by ryan0980
2 tasks done
@tonysy
[Bug] The relative path in tools/list_configs.py
#1873 opened Feb 15, 2025 by Sibyl233
2 tasks done
@tonysy
2
[Feature] expose max_task_size in run.py, for quick debug
#1869 opened Feb 13, 2025 by yxdyc
1 task
@acylam
[Feature] 是否可以有输出结果后，单独去计算一下评分的功能
#1866 opened Feb 12, 2025 by Dagoli
1 task
@MaiziXiao
1
[Feature] 请问如何加载本地模型进行测评
#1865 opened Feb 11, 2025 by Castrol68
1 task
@liushz
ProTip! Find all open issues with in progress development work with linked:pr.