Skip to content

t5gemma

T5-Gemma encoder wrapper for daVinci-MagiHuman.

MagiHuman uses transformers.models.t5gemma.T5GemmaEncoderModel on google/t5gemma-9b-9b-ul2 (a gated Google repo). This wrapper follows the same lazy-loading pattern as fastvideo/models/encoders/gemma.py: we keep the HF module under self._t5gemma_model and exclude it from named_parameters so FastVideo's weight loader does not try to load encoder shards from the converted repo directory.

For the base MagiHuman T2V port there are no additional connector layers on top — the pipeline prompt-preprocessing stage handles pad-or-trim to text_len and exposes both the padded embedding and the original length.

Classes

fastvideo.models.encoders.t5gemma.T5GemmaEncoderModel 

T5GemmaEncoderModel(config: TextEncoderConfig)

Bases: TextEncoder

Thin wrapper over HuggingFace's T5GemmaEncoderModel.

On first forward, the wrapper lazily instantiates the upstream encoder from t5gemma_model_path (defaulting to google/t5gemma-9b-9b-ul2). Afterwards, forward returns a BaseEncoderOutput with last_hidden_state = [B, L, 3584] matching MagiHuman's context.half() output.

Source code in fastvideo/models/encoders/t5gemma.py 
42
43
44
45
46
47
def __init__(self, config: TextEncoderConfig) -> None:
    super().__init__(config)
    arch = config.arch_config
    self.t5gemma_model_path: str = arch.t5gemma_model_path
    self.t5gemma_dtype: str = arch.t5gemma_dtype
    self._t5gemma_model = None