Skip to content

swiglu

MiniMax H3's value-first packed SwiGLU fusion.

Functions:

fastvideo.models.dits.minimax_h3_fusions.swiglu.minimax_h3_swiglu 

minimax_h3_swiglu(x: Tensor) -> Tensor

Run the forward-only Triton fusion over an H3 (..., 2 * ffn_dim) input.

This is intentionally a strict kernel wrapper: callers own fallback policy and must only invoke it for a supported CUDA inference path.

Source code in fastvideo/models/dits/minimax_h3_fusions/swiglu.py 
113
114
115
116
117
118
119
120
121
122
123
def minimax_h3_swiglu(x: torch.Tensor) -> torch.Tensor:
    """Run the forward-only Triton fusion over an H3 ``(..., 2 * ffn_dim)`` input.

    This is intentionally a strict kernel wrapper: callers own fallback policy and
    must only invoke it for a supported CUDA inference path.
    """
    if torch.is_grad_enabled() and x.requires_grad:
        raise RuntimeError("MiniMax H3 fused SwiGLU is forward-only and does not implement autograd")
    if torch.compiler.is_compiling():
        return torch.ops.fastvideo._minimax_h3_swiglu(x)
    return _minimax_h3_swiglu_impl(x)