AI Training Datasets

Copyrighted books from shadow libraries

5.85 billion image-text pairs from the web

Library Genesis, a shadow library of pirated books and articles.

An 825 GiB dataset of datasets

Unpublished books from Smashwords

Over 100,000 podcast episodes and transcripts

Billions of videos and their transcripts

Billions of web pages from the internet

Billions of lines of public code from GitHub

Open Super-large Crawled Aggregated coRpus, a multilingual dataset from Common Crawl.

The BigScience ROOTS Corpus, a large, documented, multilingual dataset.

High-quality text from Reddit links

A colossal, cleaned version of Common Crawl.

12.8B image-text pairs for research

Flickr-Faces-HQ: 70,000 high-quality face images.

Pirate Library Mirror, a major mirror of shadow libraries.

Filtered web data to train Falcon models

Pre-print scientific papers

A corpus of all articles from Wikipedia