Andrej Karpathy 3ba42e8135 Fix SDPA KV-cache decode to respect sliding window (#456) ...

SDPA fallback now respects sliding window during single-token KV-cache
decode by slicing K/V to the last (window + 1) tokens.

Also simplifies the mask building for chunk inference to properly apply
sliding window in that path as well.

Fixes #452

Co-Authored-By: Kartik Vashishta <kartikv776@gmail.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-01-30 17:32:12 +00:00

..

__init__.py

initial commit

2025-10-13 06:49:24 -07:00

checkpoint_manager.py

minor helpful message

2026-01-15 03:20:21 +00:00

common.py

i think this is the new torch 2.9+ API for declaring tf32 preference

2026-01-30 17:03:15 +00:00

core_eval.py

initial commit

2025-10-13 06:49:24 -07:00

dataloader.py

contiguous views and single HtoD transfer for inputs/targets much cleaner

2026-01-30 00:23:01 +00:00

dataset.py

initial commit

2025-10-13 06:49:24 -07:00

engine.py

fix: pass device_type to compute_init in engine.__main__ (#451)

2026-01-19 17:19:51 -08:00

execution.py

nit delete redundant catch/raise in execute

2025-10-29 08:10:03 -07:00

flash_attention.py

Fix SDPA KV-cache decode to respect sliding window (#456)

2026-01-30 17:32:12 +00:00

gpt.py

Fix generate() crash when top_k=0 (#467)

2026-01-30 09:21:02 -08:00

logo.svg

initial commit

2025-10-13 06:49:24 -07:00

loss_eval.py

fix typos

2025-11-14 11:20:25 +01:00

optim.py

Combine AdamW and Muon into single MuonAdamW optimizer, cleaner, ty @chrisjmccormick for idea/help

2026-01-29 00:52:08 +00:00

report.py

fix small bug where this would break if git stage has deleted files

2026-01-04 19:11:43 +00:00

tokenizer.py

adjust the comment on the regex pattern per recent experimnet see dev/LOG.md

2026-01-13 17:50:39 +00:00

ui.html

Fix conversation scroll to bottom on some browsers + remove duplicated padding (#348)

2025-12-31 13:03:22 -08:00