HYWorld (HY-WorldPlay) model components for FastVideo.
This module provides: - HYWorldTransformer3DModel: The main transformer model with ProPE and action conditioning - HYWorldVideoGenerator: Extended VideoGenerator for HYWorld inference - Utilities for pose processing and camera trajectory generation
fastvideo.models.dits.hyworld.HYWorldTransformer3DModelHYWorldTransformer3DModel(config: HYWorldConfig, hf_config: dict[str, Any])HY-World Transformer extending HunyuanVideo15 with: - ProPE (Projective Positional Encoding) for camera-aware attention - Action conditioning for interactive video generation
fastvideo.models.dits.hyworld.HYWorldTransformer3DModel.forwardforward(hidden_states: torch.Tensor, encoder_hidden_states: list[torch.Tensor], timestep: torch.LongTensor, encoder_hidden_states_image: list[torch.Tensor], encoder_attention_mask: list[torch.Tensor], action: torch.Tensor, viewmats: torch.Tensor, Ks: torch.Tensor, timestep_txt: torch.LongTensor, guidance: Optional[torch.Tensor] = None, timestep_r: Optional[torch.LongTensor] = None, attention_kwargs: Optional[dict[str, Any]] = None)Forward pass with action and camera conditioning.
action: torch.TensorAction tensor for action conditioning [B, T] or [B*T]
viewmats: torch.TensorCamera view matrices [B, T, 4, 4]
Ks: torch.TensorCamera intrinsics [B, T, 3, 3]
fastvideo.models.dits.hyworld.HYWorldDoubleStreamBlockHYWorldDoubleStreamBlock(hidden_size: int, num_attention_heads: int, mlp_ratio: float, dtype: torch.dtype | None = None, supported_attention_backends: tuple[AttentionBackendEnum, ...] | None = None, prefix: str = '')Extended MMDoubleStreamBlock with ProPE (Projective Positional Encoding) support for camera-aware attention in HY-World/WorldPlay models.
fastvideo.models.dits.hyworld.HYWorldDoubleStreamBlock.forwardforward(img: torch.Tensor, txt: torch.Tensor, encoder_attention_mask: torch.Tensor, vec: torch.Tensor, vec_txt: torch.Tensor, freqs_cis: tuple, original_seq_len: int, viewmats: torch.Tensor, Ks: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]Forward pass with ProPE camera conditioning.
img: torch.TensorImage/video tokens
txt: torch.TensorText tokens
encoder_attention_mask: torch.TensorText attention mask
vec: torch.TensorModulation vector
freqs_cis: tupleRotary embedding frequencies
original_seq_len: intOriginal (unpadded) image sequence length
viewmats: torch.TensorCamera view matrices for ProPE [B, T, 4, 4]
Ks: torch.TensorCamera intrinsics for ProPE [B, T, 3, 3]
tuple[torch.Tensor, torch.Tensor]Tuple of (img, txt) output tokens
fastvideo.models.dits.hyworld.get_resolution_from_imageget_resolution_from_image(image_path, target_resolution = '480p')Automatically determine resolution from input image.
image_pathPath or URL to input image
target_resolutionTarget resolution tier ("480p", "720p", etc.)
tuple[int, int]: (height, width) matching HY-WorldPlay's bucket selection