Dataset Viewer
The dataset viewer is not available for this dataset.
Unexpected token '<', "<html>
<h"... is not valid JSON
Need help to make the dataset viewer work? Make sure to review how to configure the dataset viewer, and open a discussion for direct support.
snimu/finemath-4plus-tiktokenized
This dataset contains GPT-2 tokenized shards of the FineWeb-4plus dataset. Each shard is stored in a binary format with the following structure:
- First comes a header with 256 int32s
- The tokens follow, each as uint16 (GPT-2 format)
The first shard is the validation set, subsequent shards are training data.
Original dataset: HuggingFaceTB/finemath
- Downloads last month
- 21