Skip to content

mmaudio_preprocess_pipeline

Native FastVideo preprocessing pipeline for MMAudio training.

Classes

fastvideo.pipelines.preprocess.mmaudio.mmaudio_preprocess_pipeline.MMAudioPreprocessPipeline 

MMAudioPreprocessPipeline(model_path: str, fastvideo_args: FastVideoArgs, **kwargs)

Bases: ComposedPipelineBase

Encode raw audio, video, and captions into MMAudio training features.

Source code in fastvideo/pipelines/preprocess/mmaudio/mmaudio_preprocess_pipeline.py 
23
24
25
26
27
28
29
30
def __init__(self, model_path: str, fastvideo_args: FastVideoArgs, **kwargs) -> None:
    # Official training features are extracted in fp32. These overrides
    # are local to preprocess mode and do not change inference defaults.
    config = fastvideo_args.pipeline_config
    config.text_encoder_precisions = ("fp32", )
    config.image_encoder_precisions = ("fp32", "fp32")
    config.audio_encoder_precision = "fp32"
    super().__init__(model_path, fastvideo_args, **kwargs)