Please note that truthfulqa is the task name and mc1 is the metrics. You should use input truthfulqa instead of truthfulqa_mc1. The trutufulqa task will output metrics mc1 and mc2. Huggingface leaderboard https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard, it uses mc2 as the...
MC1 (Single-true): Given a question and 4-5 answer choices, select the only correct answer. The model's selection is the answer choice to which it assigns the highest log-probability of completion following the question, independent of the other answer choices. The score is the simple accura...
MC1 (Single-true): Given a question and 4-5 answer choices, select the only correct answer. The model's selection is the answer choice to which it assigns the highest log-probability of completion following the question, independent of the other answer choices. The score is the simple accura...
MC1 (Single-true): Given a question and 4-5 answer choices, select the only correct answer. The model's selection is the answer choice to which it assigns the highest log-probability of completion following the question, independent of the other answer choices. The score is the simple accura...