Tristan/RedPajama-Data-V2-sample-100B-filtered-shuffled-tokenized-with-token-counts Viewer β’ Updated May 31, 2024 β’ 4.16M β’ 493
Tristan/RedPajama-Data-V2-sample-100B-filtered-for-regression-domains-with-domains Viewer β’ Updated May 24, 2024 β’ 4.16M β’ 255
Tristan/olm-wikipedia-20221220-1-percent-tokenized-766 Viewer β’ Updated Jan 18, 2023 β’ 65.1k β’ 14
Tristan/olm-wikipedia-20221220-1-percent-tokenized-568 Viewer β’ Updated Jan 17, 2023 β’ 87.8k β’ 9
Tristan/t5-small-october-wikipedia-2022-tokenized-512 Viewer β’ Updated Jan 2, 2023 β’ 9.74M β’ 979
Tristan/wikipedia-august-october-line-diff-1000-char-threshold-1000-sample Viewer β’ Updated Dec 13, 2022 β’ 1k β’ 25
Tristan/wikipedia-august-october-line-diff-1000-char-threshold Viewer β’ Updated Dec 13, 2022 β’ 286k β’ 17
Tristan/olm-october-2022-tokenized-1024-suffix-array-dedup Viewer β’ Updated Dec 11, 2022 β’ 13.2M β’ 2.18k
Tristan/olm-CC-MAIN-2022-40-sampling-ratio-0.15894621295-suffix-array-dedup Viewer β’ Updated Dec 10, 2022 β’ 7.52M β’ 988