fam/nanochat-omni
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

move eval bundle download to be lazy and inside the python code so that we can substantially simplify the run bash scripts

Browse Source
This commit is contained in:
Andrej Karpathy
2025-11-01 16:04:38 +00:00
parent 7d2c4a3d95
commit cf587acb1a
5 changed files with 35 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files
speedrun.sh
-9
View File
@@ -73,15 +73,6 @@ python -m scripts.tok_eval
# -----------------------------------------------------------------------------
# Base model (pretraining)
# Download the eval_bundle from s3 to evaluate CORE metric during training (~162MB)
EVAL_BUNDLE_URL=https://karpathy-public.s3.us-west-2.amazonaws.com/eval_bundle.zip
if [ ! -d "$NANOCHAT_BASE_DIR/eval_bundle" ]; then
curl -L -o eval_bundle.zip $EVAL_BUNDLE_URL
unzip -q eval_bundle.zip
rm eval_bundle.zip
mv eval_bundle $NANOCHAT_BASE_DIR
fi
# The d20 model is 561M parameters.
# Chinchilla says #tokens = 20X #params, so we need 561e6 * 20 = 11.2B tokens.
# Assume our tokenizer is 4.8 chars/token, this is 11.2B * 4.8 ~= 54B chars.