TreeAILab/Multi-turn_Long-context_Benchmark_for_LLMs

LoopServe: An Adaptive Dual-phase LLM Inference Acceleration System for Multi-Turn Dialogues

Arxiv: https://www.arxiv.org/abs/2507.13681

Huggingface: https://huggingface.co/papers/2507.13681

Introduction

LoopServe Multi-Turn Dialogue Benchmark is a comprehensive evaluation dataset comprising multiple diverse datasets designed to assess large language model performance in realistic conversational scenarios. Unlike traditional benchmarks that place queries only at the end of input sequences, this benchmark features diverse query positions (beginning, middle, end) across multi-turn conversations, spanning Question Answering, Needle in a haystack, Summarization, and Few-shot Learning tasks. The benchmark captures the dynamic dependencies and unpredictable patterns characteristic of real-world multi-turn dialogues to provide more authentic LLM evaluation in practical conversational applications.

Dataset statistics

For more details, please refer to our paper.

Sample Usage

The dataset can be easily loaded using the load_dataset function from the 🤗 Datasets library.

from datasets import load_dataset

# Load a specific configuration, for example, the multi-turn question answering data
dataset = load_dataset("TreeAILab/Multi-turn_Long-context_Benchmark_for_LLMs", "multi-turn_QA")

# Access the training split
print(dataset["train"])

# Iterate through an example
for example in dataset["train"]:
    print(example)
    break

Citation

@misc{li2025loopserveadaptivedualphasellm,
      title={LoopServe: An Adaptive Dual-phase LLM Inference Acceleration System for Multi-Turn Dialogues}, 
      author={Haoyang Li and Zhanchao Xu and Yiming Li and Xuejia Chen and Darian Li and Anxin Tian and Qingfa Xiao and Cheng Deng and Jun Wang and Qing Li and Lei Chen and Mingxuan Yuan},
      year={2025},
      eprint={2507.13681},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2507.13681}, 
}