Andrej Karpathy 3ba42e8135 Fix SDPA KV-cache decode to respect sliding window (#456) ...

SDPA fallback now respects sliding window during single-token KV-cache
decode by slicing K/V to the last (window + 1) tokens.

Also simplifies the mask building for chunk inference to properly apply
sliding window in that path as well.

Fixes #452

Co-Authored-By: Kartik Vashishta <kartikv776@gmail.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-01-30 17:32:12 +00:00

..

test_attention_fallback.py

Fix SDPA KV-cache decode to respect sliding window (#456)

2026-01-30 17:32:12 +00:00

test_engine.py

update the CPU/MPS script to give reasonable results. The model can at least answer that Paris is the capital of France and knows that the sky is blue, for about 40 minutes of training on my macbook. Also fixed a bug that existed due to KVCache bfloat16 dtype assumption

2026-01-17 12:27:30 -08:00