Dataset Viewer
The dataset viewer is not available for this dataset.
Unexpected token '<', "<html> <h"... is not valid JSON

Need help to make the dataset viewer work? Make sure to review how to configure the dataset viewer, and open a discussion for direct support.

Dataset Sources

Dataset Description

BenchMAX_Domain_Translation is a dataset of BenchMAX, which evaluates the translation capability on specific domains.

We collect the domain multi-way parallel data from other tasks in BenchMAX, such as math data, code data, etc. Each sample contains one or three human-annotated translations.

Usage

git clone --recurse-submodules https://github.com/CONE-MT/BenchMAX.git
cd BenchMAX
pip install -r requirements.txt

cd tasks/translation
tasks=("ifeval" "gpqa" "lcb_v4" "mgsm" "humaneval" "nexus" "arenahard")
max_tokens_list=(512 3072 2048 1024 1024 512 3072)
for i in "${!tasks[@]}"; do
    task=${tasks[$i]}
    max_tokens=${max_tokens_list[$i]}
    python generate_translation.py -s en -t zh,es,fr,de,ru,ja,th,sw,bn,te,ar,ko,vi,cs,hu,sr --task-name $task --model-name ${model} --infer-backend vllm --max-tokens ${max_tokens}
    python generate_translation.py -s zh,es,fr,de,ru,ja,th,sw,bn,te,ar,ko,vi,cs,hu,sr -t en --task-name $task --model-name ${model} --infer-backend vllm --max-tokens ${max_tokens}

    python evaluate_translation.py -s en -t zh,es,fr,de,ru,ja,th,sw,bn,te,ar,ko,vi,cs,hu,sr --task-name $task --model-name ${model} --metrics spBLEU
    python evaluate_translation.py -s zh,es,fr,de,ru,ja,th,sw,bn,te,ar,ko,vi,cs,hu,sr -t en --task-name $task --model-name ${model} --metrics spBLEU
done

Supported Languages

Arabic, Bengali, Chinese, Czech, English, French, German, Hungarian, Japanese, Korean, Serbian, Spanish, Swahili, Telugu, Thai, Russian, Vietnamese

Citation

If you find our dataset helpful, please cite this paper:

@article{huang2025benchmax,
  title={BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models},
  author={Huang, Xu and Zhu, Wenhao and Hu, Hanxu and He, Conghui and Li, Lei and Huang, Shujian and Yuan, Fei},
  journal={arXiv preprint arXiv:2502.07346},
  year={2025}
}
Downloads last month
166

Collection including LLaMAX/BenchMAX_Domain_Translation

Paper for LLaMAX/BenchMAX_Domain_Translation