Memory
Sliding-window attention limits each token to keys and values within a recent fixed-size window. Cache growth becomes bounded for layers that use the window.
A bounded window controls memory use as sequences grow. Information beyond one window can propagate through stacked layers or through other attention patterns in the model.
The runtime evicts positions that fall outside each layer's active window. Window size, layer pattern, and model training determine the quality and memory tradeoff.
Sliding-window attention limits each token to keys and values within a recent fixed-size window. Cache growth becomes bounded for layers that use the window. The runtime evicts positions that fall outside each layer's active window. Window size, layer pattern, and model training determine the quality and memory tradeoff.
A bounded window controls memory use as sequences grow. Information beyond one window can propagate through stacked layers or through other attention patterns in the model.
RunInfraby RightNow© 2026 RunInfra. All rights reserved.
