fp8_qat_train_config ¶
FP8 (e4m3) quantization-aware training linear method (straight-through estimator).
Mirror of nvfp4_qat_train_config.py but for FP8. The weight stays a trainable bf16/fp32 master that is fake-quantized to FP8 on every forward, with a full-precision backward (STE), so the model learns to absorb FP8 linear error.
The STE lives in fastvideo.layers.fp8linear._LinearFWD8BWD16Fn (FP8 forward via torch._scaled_mm on sm89+, with a bf16 fake-quant fallback on older GPUs; full-precision backward). This method bridges it into the standard quant_config path, so it activates via transformer_quant="fp8_qat_train" on the same Wan-2.1 layers as the FP4 path (to_q/k/v/out + ffn). No conversion is needed: the weight is kept in full precision and quantized on the fly each step.
Unlike the FP4 path this needs no flashinfer and runs on any sm89+ GPU (and even older ones via the bf16 fallback), not just Blackwell.