Sofie Van Landeghem 9822cc7424 use nn.init and initialize smear gate's weight as well

2026-04-13 14:03:18 +02:00

..

__init__.py

initial commit

2025-10-13 06:49:24 -07:00

checkpoint_manager.py

tune the data mixture a bit, load optimizer by default when SFT. These were confirmed to be best settings from sweeps of sft

2026-02-18 15:49:18 +00:00

common.py

delete autocast, an unnecessary thorn in my side, manage dtypes directly

2026-03-04 23:55:30 +00:00

core_eval.py

initial commit

2025-10-13 06:49:24 -07:00

dataloader.py

big, breaking change but large upside: swap previous FineWeb-EDU dataset to NVIDIA ClimbMix dataset. Requires people to download the data shards. The upside is that training GPT-2 capablity model now only takes ~2 hours, down from 2.76 hours, so this is a huge win data-wise

2026-03-04 19:47:12 +00:00

dataset.py

big, breaking change but large upside: swap previous FineWeb-EDU dataset to NVIDIA ClimbMix dataset. Requires people to download the data shards. The upside is that training GPT-2 capablity model now only takes ~2 hours, down from 2.76 hours, so this is a huge win data-wise

2026-03-04 19:47:12 +00:00

engine.py

Autoresearch round 2: smear, backout, and hyperparameter tuning

2026-03-14 17:03:06 +00:00

execution.py

nit delete redundant catch/raise in execute

2025-10-29 08:10:03 -07:00

flash_attention.py

delete autocast, an unnecessary thorn in my side, manage dtypes directly

2026-03-04 23:55:30 +00:00

fp8.py

delete autocast, an unnecessary thorn in my side, manage dtypes directly

2026-03-04 23:55:30 +00:00

gpt.py

use nn.init and initialize smear gate's weight as well

2026-04-13 14:03:18 +02:00

logo.svg

initial commit

2025-10-13 06:49:24 -07:00

loss_eval.py

fix typos

2025-11-14 11:20:25 +01:00

optim.py

use COMPUTE_DTYPE-aware cast in Muon polar express step

2026-03-25 20:19:14 +00:00

report.py

remove leftover mid references (#491)

2026-02-02 08:33:46 -08:00

tokenizer.py

adjust the comment on the regex pattern per recent experimnet see dev/LOG.md

2026-01-13 17:50:39 +00:00

ui.html

Fix conversation scroll to bottom on some browsers + remove duplicated padding (#348)

2025-12-31 13:03:22 -08:00