arjunguha/mbpp

This is the MBPP dataset. Downloaded from here and constructed as follows:

import datasets
ds = datasets.load_dataset("json", data_files="mbpp.jsonl", split="train")
test = ds.filter(lambda item: item['task_id'] >= 11 and item['task_id'] <= 510)
few_shot = ds.filter(lambda item: item['task_id'] >= 1 and item['task_id'] <= 10)
validation = ds.filter(lambda item: item['task_id'] >= 511 and item['task_id'] <= 600)
train = ds.filter(lambda item: item['task_id'] >= 601 and item['task_id'] <= 974)
ds = datasets.DatasetDict({ "train": train, "few_shot": few_shot, "validation": validation, "test": test })
ds.push_to_hub("arjunguha/mbpp")

Credit:

@misc{austin2021programsynthesislargelanguage,
      title={Program Synthesis with Large Language Models}, 
      author={Jacob Austin and Augustus Odena and Maxwell Nye and Maarten Bosma and Henryk Michalewski and David Dohan and Ellen Jiang and Carrie Cai and Michael Terry and Quoc Le and Charles Sutton},
      year={2021},
      eprint={2108.07732},
      archivePrefix={arXiv},
      primaryClass={cs.PL},
      url={https://arxiv.org/abs/2108.07732}, 
}