Quantization
Channelwise quantization assigns a separate scale to each channel instead of sharing one scale across an entire tensor. The finer scale granularity follows differences in magnitude between channels.
Per-channel scales can preserve useful signal when a few channels have much larger values than others. They add scale metadata and may require kernels that apply those scales efficiently.
The quantizer selects a channel axis, computes scales along it, and stores them beside the quantized values. Serving kernels must read the same layout and apply matching dequantization or low-precision math.
Channelwise quantization assigns a separate scale to each channel instead of sharing one scale across an entire tensor. The finer scale granularity follows differences in magnitude between channels. The quantizer selects a channel axis, computes scales along it, and stores them beside the quantized values. Serving kernels must read the same layout and apply matching dequantization or low-precision math.
Per-channel scales can preserve useful signal when a few channels have much larger values than others. They add scale metadata and may require kernels that apply those scales efficiently.
RunInfraby RightNow© 2026 RunInfra. All rights reserved.
