PathGen-1.6M: 1.6 Million Pathology Image-text Pairs Generation through Multi-agent Collaboration
Paper • 2407.00203 • Published • 1
Datasets:
Dataset Viewer
The dataset viewer is not available for this subset.
Cannot get the split names for the config 'default' of the dataset.
Exception: SplitsNotFoundError
Message: The split names could not be parsed from the dataset config.
Traceback: Traceback (most recent call last):
File "/usr/local/lib/python3.14/site-packages/datasets/inspect.py", line 286, in get_dataset_config_info
for split_generator in builder._split_generators(
~~~~~~~~~~~~~~~~~~~~~~~~~^
StreamingDownloadManager(base_path=builder.base_path, download_config=download_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/local/lib/python3.14/site-packages/datasets/packaged_modules/hdf5/hdf5.py", line 49, in _split_generators
import h5py
ModuleNotFoundError: No module named 'h5py'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/src/services/worker/src/worker/job_runners/config/split_names.py", line 66, in compute_split_names_from_streaming_response
for split in get_dataset_split_names(
~~~~~~~~~~~~~~~~~~~~~~~^
path=dataset,
^^^^^^^^^^^^^
config_name=config,
^^^^^^^^^^^^^^^^^^^
token=hf_token,
^^^^^^^^^^^^^^^
)
^
File "/usr/local/lib/python3.14/site-packages/datasets/inspect.py", line 340, in get_dataset_split_names
info = get_dataset_config_info(
path,
...<6 lines>...
**config_kwargs,
)
File "/usr/local/lib/python3.14/site-packages/datasets/inspect.py", line 291, in get_dataset_config_info
raise SplitsNotFoundError("The split names could not be parsed from the dataset config.") from err
datasets.inspect.SplitsNotFoundError: The split names could not be parsed from the dataset config.Need help to make the dataset viewer work? Make sure to review how to configure the dataset viewer, and open a discussion for direct support.
WSI Classification Dataset for AEM
Dataset Summary
This dataset is derived from the publicly available CAMELYON16 and CAMELYON17 datasets. It consists of feature embeddings extracted from tissue patches of whole slide images (WSIs) using various pre-trained models. The dataset is designed for use in multiple instance learning (MIL) based WSI classification tasks, particularly for the Attention Entropy Maximization (AEM) method.
Usage
For detailed instructions on using this dataset with the Attention Entropy Maximization (AEM) method, please refer to the official AEM GitHub repository and arXiv paper:
- AEM GitHub Repository
- AEM: Attention Entropy Maximization for Multiple Instance Learning based Whole Slide Image Classification
These resources provide implementation details, examples, and documentation on applying AEM to WSI classification tasks using this dataset.
Dataset Creation
Source Data
- CAMELYON16: 400 WSIs of sentinel lymph node sections. More info
- CAMELYON17: 500 WSIs with slide-level annotations, selected from the CAMELYON17 training set. More information
Data Processing
Tissue patches were extracted from the WSIs using the CLAM toolkit.
Feature embeddings were generated for each patch using these pre-trained models:
- ResNet18: PyTorch pre-trained weights
- Lunit pre-trained DINO: Checkpoint
- PathGen-CLIP: Pre-trained checkpoints
Considerations for Using the Data
Intended Uses
This dataset is primarily intended for research in computational pathology, specifically for developing and evaluating MIL-based WSI classification methods.
Social Impact and Biases
While this dataset aims to advance research in computational pathology and potentially improve diagnostic tools, users should be aware of potential biases inherent in the original CAMELYON datasets. These biases may affect the generalizability of models trained on this data.
Additional Information
Licensing Information
This dataset is released under the Apache 2.0 license.
Citation Information
If you use this dataset, please cite:
@article{zhang2023attention,
title={Attention-challenging multiple instance learning for whole slide image classification},
author={Zhang, Yunlong and Li, Honglin and Sun, Yuxuan and Zheng, Sunyi and Zhu, Chenglu and Yang, Lin},
journal={arXiv preprint arXiv:2311.07125},
year={2023}
}
@misc{zhang2024aemattentionentropymaximization,
title={AEM: Attention Entropy Maximization for Multiple Instance Learning based Whole Slide Image Classification},
author={Yunlong Zhang and Zhongyi Shui and Yunxuan Sun and Honglin Li and Jingxiong Li and Chenglu Zhu and Lin Yang},
year={2024},
eprint={2406.15303},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2406.15303}
}
- Downloads last month
- 35