Andrej Karpathy b07604ebaa document the legacy fineweb100b dataset and the new climbmix400b dataset

2026-03-03 17:24:31 +00:00

..

estimate_gpt3_core.ipynb

add notebook on deriving the CORE estimates for the GPT-3 miniseries.

2026-01-05 18:40:28 +00:00

gen_synthetic_data.py

tune the synthetic data generation script. delete the king andrej stuff lol. also, upgrade to gemini 3

2026-02-02 01:45:59 +00:00

generate_logo.html

initial commit

2025-10-13 06:49:24 -07:00

LEADERBOARD.md

new leaderboard record via new auto-calculated optimal batch size. for d26 it is 1M, up from 0.5M that was default earlier

2026-02-05 20:11:32 +00:00

LOG.md

tune logit softcap?

2026-03-03 00:38:53 +00:00

nanochat.png

Update logo

2025-10-14 14:19:44 -04:00

repackage_data_reference.py

document the legacy fineweb100b dataset and the new climbmix400b dataset

2026-03-03 17:24:31 +00:00

scaling_analysis.ipynb

add engram-lite, add log, tune scaling laws analysis scripts

2026-01-27 22:31:17 +00:00

scaling_laws_jan26.png

nuke midtraining from orbit, it's not as needed now that we have a BOS-aligned dataloader. Also change the README a lot. midtrianing is not yet fully properly erased across the board, but good enough for step 1

2026-01-31 19:12:25 +00:00