SANA-Video 推理实战指南CLI 批量生成与 Sana LTX2 两阶段精修管线【免费下载链接】SanaSANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer项目地址: https://gitcode.com/GitHub_Trending/sana/Sana本文面向希望在本仓库Sana中运行 SANA-Video 视频生成推理的开发者系统讲解从单条 Prompt 的 CLI 推理、TXT 批量文生视频/图生视频脚本到Sana 生成 latent → LTX2 精修两阶段高清视频管线2K 质量、720p 延迟的完整落地方法。阅读本文后你将掌握 480p / 720p 两套官方配置的用法、所有推理命令行参数的真实含义以及基于 diffusers 的SanaVideoPipeline与 LTX2 Refiner 的端到端集成代码。SANA-Video 推理概览SANA-Video 是本仓库中的高效视频生成模型采用Block Linear Diffusion Transformer架构核心算子为线性注意力Linear Attention配合 Block Causal 的常内存 KV 缓存设计可以在消费级 GPU如 RTX 5090上以较低成本生成最长分钟级、最高 2K 分辨率的高质量视频。官方能力与性能陈述请以 docs/sana_video.md 为准本文聚焦其推理侧的工程细节所有命令与参数均可在当前仓库中找到对应的源码、配置或测试佐证。从推理形态上本仓库提供两条主线原生推理脚本基于pyrallis配置解析 自研 samplerFlowEuler、DPMS、LongLiveFlowEuler、FastVideoDMD4Step等以accelerate launch方式在单机多卡上批量生成视频diffusers 两阶段管线先用SanaVideoPipeline输出视频 latent再用 LTX-2 Stage-2蒸馏 3 步做空间分辨率精修最终得到 2K 级别画面。准备工作模型权重与配置文件推理前需要准备两样东西模型权重通过hf://协议路径加载与YAML 配置文件由pyrallis解析。当前仓库configs/sana_video_config/目录下与推理直接相关的配置包括配置分辨率/VAE说明Sana_2000M_480px_AdamW_fsdp.yaml480p / WanVAE基础文生视频/图生视频配置SanaMSVideo_2000M_P2_D20模型Sana_2000M_480px_AdamW_fsdp_chunk.yaml480p / WanVAEChunked 推理版本配合--interval_k与首帧图条件使用Sana_2000M_720px_ltx2vae_AdamW_fsdp.yaml720p / LTX2VAE720p 高清版本SanaMSVideo_2000M_P1_D20patch_size1官方权重地址约定使用hf://协议例如hf://Efficient-Large-Model/SANA-Video_2B_480p/checkpoints/SANA_Video_2B_480p.pth。加载逻辑实现在 tools/download.py 的find_model中inference_video_scripts/inference_sana_video.py 会调用它并做 state_dict 归一化自动剥掉generator/state_dict_ema/model.前缀等包装见_normalize_model_state_dict。单条 Prompt 推理 CLI原文档给出的单条文本生成命令如下注意当前仓库快照中app/目录未包含sana_video_pipeline.py其实际批量落地入口见下一节的inference_sana_video.sh配置名亦以仓库实际文件为准python app/sana_video_pipeline.py \ --config configs/sana_video_config/480ms/Sana_1600M_480px_adamW_fsdp.yaml \ --model_path hf://Efficient-Large-Model/SanaVideo_willquant/checkpoints/model.pth \ --save_path sana_video.mp4 \ --prompt In a whimsical forest setting, a small deer with antlers stands amidst oversized mushrooms and scattered carrots. The scene is vibrant with lush green moss and rocks, creating a magical atmosphere. The deer appears curious, moving slowly across the ground, surrounded by the towering fungi and colorful vegetables. The sky above is clear and bright, adding to the enchanting ambiance. A low-angle shot captures the deers gentle exploration of this fantastical landscape.Chunked 版本图生视频Chunked 推理版本在文档中用于以图像为首帧条件的图生视频关键参数是--image_path条件首帧与--interval_k控制首帧条件的噪声强度/时间步权重python app/sana_video_pipeline.py \ --config configs/sana_video_config/480ms/Sana_1600M_480px_adamW_fsdp_chunk.yaml \ --model_path hf://Efficient-Large-Model/SanaVideo_chunk/checkpoints/model.pth \ --save_path sana_video_chunk_i2v.mp4 \ --interval_k 0.2 \ --image_path output/tmp_videos/wan_goodcase_i2v_eval/00000000_video_001.jpg \ --prompt In a whimsical forest setting, a small deer with antlers stands amidst oversized mushrooms and scattered carrots. ...仓库中与此对应的 chunk 配置为 Sana_2000M_480px_AdamW_fsdp_chunk.yaml--interval_k在推理脚本中会写入输出目录后缀_interval_k{int(interval_k*1000)}用于标识不同的条件强度实验。批量推理inference_sana_video.sh当前仓库真正可运行的批量推理入口是 inference_video_scripts/inference_sana_video.sh它内部会执行accelerate launch --num_processes$np --num_machines1 --mixed_precisionbf16 --main_process_port$RANDOM \ inference_video_scripts/inference_sana_video.py \ --config$config --model_path$model_path \ --txt_fileasset/samples/video_prompts_samples.txt \ --datasetvideo_samples其中--np指定进程数其余参数会被透传给 Python 脚本shell 解析后拼接到命令末尾。请使用真实存在的配置路径文档中的configs/sana_video_config/480ms/...在当前仓库对应为 Sana_2000M_480px_AdamW_fsdp.yaml。文生视频480pbash inference_video_scripts/inference_sana_video.sh \ --np 1 \ --config configs/sana_video_config/Sana_2000M_480px_AdamW_fsdp.yaml \ --model_path hf://Efficient-Large-Model/SANA-Video_2B_480p/checkpoints/SANA_Video_2B_480p.pth \ --txt_fileasset/samples/video_prompts_samples.txt \ --cfg_scale 6 \ --motion_score 30 \ --flow_shift 8 \ --work_dir output/sana_t2v_video_results图生视频480p图生视频通过--taskltx指定提示词文件中的每一行格式为文本描述image图片路径image即源码中的image_split_token默认值见 sample_i2v.txtbash inference_video_scripts/inference_sana_video.sh \ --np 1 \ --config configs/sana_video_config/Sana_2000M_480px_AdamW_fsdp.yaml \ --model_path hf://Efficient-Large-Model/SANA-Video_2B_480p/checkpoints/SANA_Video_2B_480p.pth \ --txt_fileasset/samples/sample_i2v.txt \ --taskltx \ --cfg_scale 6 \ --motion_score 30 \ --flow_shift 8 \ --work_dir output/sana_ti2v_video_results720p 高清推理720p 模型使用 LTX-2 VAE32× 空间压缩、8× 时间压缩实现更高分辨率视频生成。配置为 Sana_2000M_720px_ltx2vae_AdamW_fsdp.yaml模型权重对应SANA_Video_2B_720p.pth# 文生视频720p bash inference_video_scripts/inference_sana_video.sh \ --np 1 \ --config configs/sana_video_config/Sana_2000M_720px_ltx2vae_AdamW_fsdp.yaml \ --model_path hf://Efficient-Large-Model/SANA-Video_2B_720p/checkpoints/SANA_Video_2B_720p.pth \ --txt_fileasset/samples/video_prompts_samples.txt \ --cfg_scale 6 \ --motion_score 30 \ --flow_shift 8 \ --work_dir output/sana_t2v_720p_results # 图生视频720p bash inference_video_scripts/inference_sana_video.sh \ --np 1 \ --config configs/sana_video_config/Sana_2000M_720px_ltx2vae_AdamW_fsdp.yaml \ --model_path hf://Efficient-Large-Model/SANA-Video_2B_720p/checkpoints/SANA_Video_2B_720p.pth \ --txt_fileasset/samples/sample_i2v.txt \ --taskltx \ --cfg_scale 6 \ --motion_score 30 \ --flow_shift 8 \ --work_dir output/sana_ti2v_720p_results推理脚本源码解析参数与执行流批量推理的核心实现位于 inference_video_scripts/inference_sana_video.py它以SanaInference(SanaVideoConfig)数据类定义全部 CLI 参数L406-L451再通过pyrallis.parse(config_classSanaInference, config_pathargs.config)将命令行参数与 YAML 配置合并。完整参数表参数默认值说明--configconfigs/sana_video_config/Sana_2000M_480px_AdamW_fsdp.yaml配置文件路径--model_pathhf://.../SANA_Video_2B_480p.pth权重路径支持hf://协议与本地.binFSDP权重--work_dir由 model_path 推导输出根目录视频保存于work_dir/vis/...--txt_file/--json_fileasset/samples/video_prompts_samples.txt/None提示词来源逐行 txt 或 JSON 字典--fps16输出视频帧率--bs1batch size--num_frames-1-1 时使用配置data.num_frames默认 81 帧--cfg_scale6.0Classifier-Free Guidance 强度--flow_shiftNone为空时回落到配置scheduler.inference_flow_shift480p 为 7.0720p 为 8.0--sampling_algoNone为空时使用配置scheduler.vis_sampler默认flow_dpm-solver--skip_typetime_uniform_flow时间步采样方式用于 DPM-Solver--guidance_typeclassifier-free可选adaptive_projected_guidance、classifier-free_STG--seed0随机种子按seed rank分配--step-1采样步数-1 时默认 50 步--motion_score10运动强度见下方 motion prompt 规则--negative_prompt反伪影长句默认即为文档中那段 chaotic sequence... 负面提示词--interval_k0.0图生视频首帧条件的强度--image_split_tokenimage图生视频 txt 中分隔文本与图片路径的 token--prompt_split_tokensplit单行多 prompt 的分隔 token--stg_applied_layers/--stg_scale[]/ 0.0时空引导STG相关--apg_modehw自适应投影引导APG模式--num_cached_blocks-1LongLive 长视频 KV 缓存块数motion prompt 拼接规则源码 L497-L504 定义了运动提示词的生成逻辑motion_score 0在 prompt 末尾追加 motion score: N.如motion score: 30.motion_score 0不追加任何运动提示motion_score 0根据--high_motion追加 high motion或 low motion使用 FastVideoDMD 4 步采样时仅支持high motion。采样器分支visualize函数根据sampling_algo分发到不同 solverL302-L379flow_euler标准 Flow Euler支持 APG 引导flow_dpm-solverDPM-Solver 二阶多步配置文件默认vis_samplerflow_euler_ltx图生视频专用--taskltx/ti2v时自动切换将首帧图像编码进潜空间作为条件帧longlive_flow_euler长视频num_frames超过模型基础帧数时必须启用要求--cfg_scale 1.0结合--num_cached_blocks使用fastvideo_dmd_4step4 步快速采样仅支持规范 T2V可配合--generator_sigma_profile与--high_motion。图生视频的底层实现当config.task ltx时脚本会用vae_encode将输入图像编码为 latent写入噪声张量第 0 帧condition_frame_info {0: 0.0}其余帧继续去噪从而形成首帧条件L264-L300。若配置了图像编码器image_encoder还会额外注入图像 embedding 参与跨模态条件。配置解析480p 与 720p 的关键差异对比两份推理配置可以清楚看到两个分辨率版本在架构上的分工维度480pWanVAE720pLTX2VAE模型SanaMSVideo_2000M_P2_D20patch_size2SanaMSVideo_2000M_P1_D20patch_size1VAEWanVAE16 通道8× 空间下采样stride[4,8,8]LTX2VAE_diffusersAutoencoderKLLTX2Video128 通道32× 空间压缩stride[8,32,32]VAE 精度float32bfloat16宽高比表ASPECT_RATIO_VIDEO_480_MSASPECT_RATIO_VIDEO_720_MS_DIV32保证空间维度被 32 整除文本编码器gemma-2-2b-itmodel_max_length300带 CHI 提示词增强同左flow_shift / inference_flow_shift3.0 / 7.03.0 / 8.0720p 之所以改用 patch_size1是因为 LTX2 VAE 已将空间压缩到 32×patch 不能再叠加两套配置的scheduler均采用linear_flow噪声调度、predict_flow_v: true并使用logit_normal时间步加权logit_mean0.0, logit_std1.0。这些设置在训练与推理间保持一致详见 Sana_2000M_720px_ltx2vae_AdamW_fsdp.yaml 与 docs/sana_video.md 的训练章节。Sana Video LTX2 Refiner 两阶段精修管线这是本仓库视频推理最核心的进阶玩法官方称之为Two-Stage Inference ParadigmBet Small to Win BigStage 1轻量 2B 的 SANA-Video 负责生成时间维度上的结构与运动输出 latent不直接解码Stage 1.5可选的 latent 空间上采样LTX2LatentUpsamplerModel/ patch upsampleStage 2步蒸馏的 LTX-2 Refiner仅 3 步负责空间分辨率与纹理精修以 720p 的延迟获得接近 2K 的画质。仓库中的命令行实现为 app/sana_video_refiner_pipeline_diffusers.pypython app/sana_video_refiner_pipeline_diffusers.py \ --sana_model_id Efficient-Large-Model/SANA-Video_2B_720p_diffusers \ --ltx2_model_id Lightricks/LTX-2 \ --prompt A cat and a dog baking a cake together in a kitchen. \ --sana_height 704 \ --sana_width 1280 \ --sana_frames 81 \ --output_path sana_ltx2_refined.mp4CLI 参数表参数默认值说明--prompt猫狗烘焙示例句生成提示词内部会追加motion score: N.--negative_prompt反伪影长句与原生推理脚本相同的默认负面提示词--sana_model_idEfficient-Large-Model/SANA-Video_2B_720p_diffusersStage-1 模型diffusers 格式--ltx2_model_idLightricks/LTX-2Stage-2 Refiner 基座--sana_height / --sana_width / --sana_frames704 / 1280 / 81Stage-1 生成尺寸与帧数--motion_score30运动强度--sana_guidance_scale / --sana_num_steps6.0 / 50Stage-1 的 CFG 与步数--frame_rate16.0输出帧率--seed42随机种子--output_pathsana_ltx2_refined.mp4Stage-2 最终输出--save_sana_outputsana_original.mp4同时保存 Stage-1 原始输出用于对比置空则跳过--skip_audioTrue是否跳过音频生成--skip_upsamplerTrue是否跳过 latent 上采样False 时使用latent_upsampler完整 Python 参考实现原文档给出了不依赖 CLI、完全可控的 diffusers 级参考代码完整继承如下Sana Video LTX2 Refiner: Stage 1 generate latent → Stage 2 refine (3 steps). import gc import torch from diffusers import SanaVideoPipeline, FlowMatchEulerDiscreteScheduler from diffusers.pipelines.ltx2 import LTX2Pipeline, LTX2LatentUpsamplePipeline from diffusers.pipelines.ltx2.latent_upsampler import LTX2LatentUpsamplerModel from diffusers.pipelines.ltx2.utils import STAGE_2_DISTILLED_SIGMA_VALUES from diffusers.pipelines.ltx2.export_utils import encode_video device cuda dtype torch.bfloat16 prompt A cat walking on the grass, facing the camera. negative_prompt A chaotic sequence with misshapen, deformed limbs in heavy motion blur, sudden disappearance, jump cuts, jerky movements, rapid shot changes, frames out of sync, inconsistent character shapes, temporal artifacts, jitter, and ghosting effects, creating a disorienting visual experience. motion_score 30 height, width, frames, frame_rate 704, 1280, 81, 16.0 seed 42 # ── Load all models ── sana_pipe SanaVideoPipeline.from_pretrained( Efficient-Large-Model/SANA-Video_2B_720p_diffusers, torch_dtypedtype, ) sana_pipe.text_encoder.to(dtype) sana_pipe.enable_model_cpu_offload() ltx_pipe LTX2Pipeline.from_pretrained(Lightricks/LTX-2, torch_dtypedtype) ltx_pipe.load_lora_weights( Lightricks/LTX-2, adapter_namestage_2_distilled, weight_nameltx-2-19b-distilled-lora-384.safetensors, ) ltx_pipe.set_adapters(stage_2_distilled, 1.0) ltx_pipe.vae.to(dtype) ltx_pipe.enable_model_cpu_offload() upsampler_model LTX2LatentUpsamplerModel.from_pretrained( Lightricks/ltx-2-patch-upsampler, torch_dtypedtype, ) upsampler LTX2LatentUpsamplePipeline(upsampler_model).to(device) # ── Stage 1: Sana-Video generates latent ── motion_prompt f motion score: {motion_score}. sana_out sana_pipe( promptprompt motion_prompt, negative_promptnegative_prompt, heightheight, widthwidth, framesframes, guidance_scale6, num_inference_steps50, generatortorch.Generator(devicedevice).manual_seed(seed), output_typelatent, ) sana_latent sana_out.frames del sana_pipe; gc.collect(); torch.cuda.empty_cache() # ── Stage 1.5: Upsample latent ── video_latent sana_latent.squeeze(0).permute(1, 0, 2, 3) packed upsampler(video_latent, output_typelatent).frames lF, _, lH, lW packed.shape pH lH * ltx_pipe.vae_spatial_compression_ratio pW lW * ltx_pipe.vae_spatial_compression_ratio pT (lF - 1) * ltx_pipe.vae_temporal_compression_ratio 1 dur pT / frame_rate audio_frames round(dur * ltx_pipe.audio_sampling_rate / ltx_pipe.audio_hop_length / ltx_pipe.audio_vae_temporal_compression_ratio) nch ltx_pipe.audio_vae.config.latent_channels mel ltx_pipe.audio_vae.config.mel_bins // ltx_pipe.audio_vae_mel_compression_ratio audio_latent ( ltx_pipe.audio_vae.latents_mean.unsqueeze(0).unsqueeze(0) .expand(1, audio_frames, nch * mel).to(dtypedtype, devicedevice).contiguous() .unflatten(2, (nch, mel)).permute(0, 2, 1, 3).contiguous() ) del video_latent; gc.collect(); torch.cuda.empty_cache() # ── Stage 2: LTX2 refine ── video, _ ltx_pipe( latentspacked, audio_latentsaudio_latent, promptprompt, negative_promptnegative_prompt, heightpH, widthpW, num_framespT, num_inference_steps3, noise_scaleSTAGE_2_DISTILLED_SIGMA_VALUES[0], sigmasSTAGE_2_DISTILLED_SIGMA_VALUES, guidance_scale1.0, frame_rateframe_rate, generatortorch.Generator(devicedevice).manual_seed(seed), output_typenp, return_dictFalse, ) video torch.from_numpy((video * 255).round().astype(uint8)) encode_video(video[0], fpsframe_rate, audioNone, audio_sample_rateNone, output_pathsana_ltx2_refined.mp4)四个关键技术点结合 app/sana_video_refiner_pipeline_diffusers.py 的源码注释这条管线有四个必须理解的细节Stage-1 输出 latent 而非像素SanaVideoPipeline以output_typelatent运行通过遍历输出对象的latents/video_latents/frames等字段提取 latent源码 L74-L87随后释放 Sana 模型并清理显存手动 pack latent 以跳过 diffusers 的 normalize源码使用LTX2Pipeline._pack_latents(...)手动完成 patch 打包patch_size 与 patch_size_t 取自 transformer 配置保证与官方 LTX-2 原版代码数值一致构造归一化后为零的音频 latent直接用audio_vae.latents_mean平铺构造音频 latent这样经过内部 normalize 后音频通道恰好为零等价于无音频避免生成管线因缺音频输入而报错源码 L180-L196Stage-2 使用蒸馏 3 步 sigma 调度STAGE_2_DISTILLED_SIGMA_VALUES (0.909375, 0.725, 0.421875, 0.0)num_inference_steps3、guidance_scale1.0配合stage_2_distilledLoRAltx-2-19b-distilled-lora-384.safetensors做确定性 Euler 去噪。另外仓库 diffusion/refiner/diffusers_ltx2_refiner.py 中还提供了同一套 sigma 常量L47以及面向 Sana-WM 的分块自回归精修实现block_size3、kv_max_frames11、sink_size1的滑窗 KV 缓存可作为将本管线扩展到流式/长视频场景的参考。模型转换pth → diffusers safetensors原生.pth检查点可以通过仓库转换脚本导出为 diffusers 格式以便配合SanaVideoPipeline/SanaImageToVideoPipeline使用python tools/convert_scripts/convert_sana_video_to_diffusers.py --dump_path output/SANA_Video_2B_480p_diffusers --save_full_pipeline转换脚本位于 tools/convert_scripts/convert_sana_video_to_diffusers.py--save_full_pipeline会连同 VAE、文本编码器与 scheduler 一起保存为完整 pipeline 目录。该步骤是使用 diffusers 版两阶段管线上一节的必要前置条件之一。与 LongSANA 等扩展的衔接如果你需要生成长视频分钟级原生推理脚本还集成了longlive_flow_euler采样器当--num_frames超过模型基础帧数配置data.num_frames默认 81时脚本会断言必须使用该采样器并通过--num_cached_blocks控制 KV 缓存的保留块数见 inference_video_scripts/inference_sana_video.py。官方 LongSANASANA-Video LongLive的相关性能陈述与说明见 docs/longsana.md 与 docs/sana_video.md。参考与延伸阅读docs/sana_video.mdSANA-Video 官方主文档含 diffusersSanaVideoPipeline/SanaImageToVideoPipeline用法、训练与 VBench 性能数据docs/model_zoo.md480p / 720p 模型权重清单inference_video_scripts/inference_sana_video.py批量推理脚本源码app/sana_video_refiner_pipeline_diffusers.py两阶段精修 CLI 实现diffusion/refiner/diffusers_ltx2_refiner.pyLTX-2 Refiner 的分块自回归实现asset/samples/video_prompts_samples.txt 与 asset/samples/sample_i2v.txt文生视频 / 图生视频提示词样本tests/test_sana_video2.pySanaVideo2 相关测试用例tools/convert_scripts/convert_sana_video_to_diffusers.py.pth权重转 diffusers 格式使用前提与限制说明本文所有命令均以当前仓库快照为基准配置文件路径以configs/sana_video_config/目录下的实际文件名为准app/目录中当前未包含文档早期版本的sana_video_pipeline.py若需单条推理可改用inference_video_scripts/inference_sana_video.py或 diffusers 管线。两阶段精修依赖的 diffusers 需为包含SanaVideoPipeline与 LTX-2 组件的较新版本请在安装依赖时以 docs/requirements.txt 与实际运行报错为准。【免费下载链接】SanaSANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer项目地址: https://gitcode.com/GitHub_Trending/sana/Sana创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考 SANA-Video 推理实战指南：CLI 批量生成与 Sana + LTX2 两阶段精修管线
SANA-Video 推理实战指南CLI 批量生成与 Sana LTX2 两阶段精修管线【免费下载链接】SanaSANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer项目地址: https://gitcode.com/GitHub_Trending/sana/Sana本文面向希望在本仓库Sana中运行 SANA-Video 视频生成推理的开发者系统讲解从单条 Prompt 的 CLI 推理、TXT 批量文生视频/图生视频脚本到Sana 生成 latent → LTX2 精修两阶段高清视频管线2K 质量、720p 延迟的完整落地方法。阅读本文后你将掌握 480p / 720p 两套官方配置的用法、所有推理命令行参数的真实含义以及基于 diffusers 的SanaVideoPipeline与 LTX2 Refiner 的端到端集成代码。SANA-Video 推理概览SANA-Video 是本仓库中的高效视频生成模型采用Block Linear Diffusion Transformer架构核心算子为线性注意力Linear Attention配合 Block Causal 的常内存 KV 缓存设计可以在消费级 GPU如 RTX 5090上以较低成本生成最长分钟级、最高 2K 分辨率的高质量视频。官方能力与性能陈述请以 docs/sana_video.md 为准本文聚焦其推理侧的工程细节所有命令与参数均可在当前仓库中找到对应的源码、配置或测试佐证。从推理形态上本仓库提供两条主线原生推理脚本基于pyrallis配置解析 自研 samplerFlowEuler、DPMS、LongLiveFlowEuler、FastVideoDMD4Step等以accelerate launch方式在单机多卡上批量生成视频diffusers 两阶段管线先用SanaVideoPipeline输出视频 latent再用 LTX-2 Stage-2蒸馏 3 步做空间分辨率精修最终得到 2K 级别画面。准备工作模型权重与配置文件推理前需要准备两样东西模型权重通过hf://协议路径加载与YAML 配置文件由pyrallis解析。当前仓库configs/sana_video_config/目录下与推理直接相关的配置包括配置分辨率/VAE说明Sana_2000M_480px_AdamW_fsdp.yaml480p / WanVAE基础文生视频/图生视频配置SanaMSVideo_2000M_P2_D20模型Sana_2000M_480px_AdamW_fsdp_chunk.yaml480p / WanVAEChunked 推理版本配合--interval_k与首帧图条件使用Sana_2000M_720px_ltx2vae_AdamW_fsdp.yaml720p / LTX2VAE720p 高清版本SanaMSVideo_2000M_P1_D20patch_size1官方权重地址约定使用hf://协议例如hf://Efficient-Large-Model/SANA-Video_2B_480p/checkpoints/SANA_Video_2B_480p.pth。加载逻辑实现在 tools/download.py 的find_model中inference_video_scripts/inference_sana_video.py 会调用它并做 state_dict 归一化自动剥掉generator/state_dict_ema/model.前缀等包装见_normalize_model_state_dict。单条 Prompt 推理 CLI原文档给出的单条文本生成命令如下注意当前仓库快照中app/目录未包含sana_video_pipeline.py其实际批量落地入口见下一节的inference_sana_video.sh配置名亦以仓库实际文件为准python app/sana_video_pipeline.py \ --config configs/sana_video_config/480ms/Sana_1600M_480px_adamW_fsdp.yaml \ --model_path hf://Efficient-Large-Model/SanaVideo_willquant/checkpoints/model.pth \ --save_path sana_video.mp4 \ --prompt In a whimsical forest setting, a small deer with antlers stands amidst oversized mushrooms and scattered carrots. The scene is vibrant with lush green moss and rocks, creating a magical atmosphere. The deer appears curious, moving slowly across the ground, surrounded by the towering fungi and colorful vegetables. The sky above is clear and bright, adding to the enchanting ambiance. A low-angle shot captures the deers gentle exploration of this fantastical landscape.Chunked 版本图生视频Chunked 推理版本在文档中用于以图像为首帧条件的图生视频关键参数是--image_path条件首帧与--interval_k控制首帧条件的噪声强度/时间步权重python app/sana_video_pipeline.py \ --config configs/sana_video_config/480ms/Sana_1600M_480px_adamW_fsdp_chunk.yaml \ --model_path hf://Efficient-Large-Model/SanaVideo_chunk/checkpoints/model.pth \ --save_path sana_video_chunk_i2v.mp4 \ --interval_k 0.2 \ --image_path output/tmp_videos/wan_goodcase_i2v_eval/00000000_video_001.jpg \ --prompt In a whimsical forest setting, a small deer with antlers stands amidst oversized mushrooms and scattered carrots. ...仓库中与此对应的 chunk 配置为 Sana_2000M_480px_AdamW_fsdp_chunk.yaml--interval_k在推理脚本中会写入输出目录后缀_interval_k{int(interval_k*1000)}用于标识不同的条件强度实验。批量推理inference_sana_video.sh当前仓库真正可运行的批量推理入口是 inference_video_scripts/inference_sana_video.sh它内部会执行accelerate launch --num_processes$np --num_machines1 --mixed_precisionbf16 --main_process_port$RANDOM \ inference_video_scripts/inference_sana_video.py \ --config$config --model_path$model_path \ --txt_fileasset/samples/video_prompts_samples.txt \ --datasetvideo_samples其中--np指定进程数其余参数会被透传给 Python 脚本shell 解析后拼接到命令末尾。请使用真实存在的配置路径文档中的configs/sana_video_config/480ms/...在当前仓库对应为 Sana_2000M_480px_AdamW_fsdp.yaml。文生视频480pbash inference_video_scripts/inference_sana_video.sh \ --np 1 \ --config configs/sana_video_config/Sana_2000M_480px_AdamW_fsdp.yaml \ --model_path hf://Efficient-Large-Model/SANA-Video_2B_480p/checkpoints/SANA_Video_2B_480p.pth \ --txt_fileasset/samples/video_prompts_samples.txt \ --cfg_scale 6 \ --motion_score 30 \ --flow_shift 8 \ --work_dir output/sana_t2v_video_results图生视频480p图生视频通过--taskltx指定提示词文件中的每一行格式为文本描述image图片路径image即源码中的image_split_token默认值见 sample_i2v.txtbash inference_video_scripts/inference_sana_video.sh \ --np 1 \ --config configs/sana_video_config/Sana_2000M_480px_AdamW_fsdp.yaml \ --model_path hf://Efficient-Large-Model/SANA-Video_2B_480p/checkpoints/SANA_Video_2B_480p.pth \ --txt_fileasset/samples/sample_i2v.txt \ --taskltx \ --cfg_scale 6 \ --motion_score 30 \ --flow_shift 8 \ --work_dir output/sana_ti2v_video_results720p 高清推理720p 模型使用 LTX-2 VAE32× 空间压缩、8× 时间压缩实现更高分辨率视频生成。配置为 Sana_2000M_720px_ltx2vae_AdamW_fsdp.yaml模型权重对应SANA_Video_2B_720p.pth# 文生视频720p bash inference_video_scripts/inference_sana_video.sh \ --np 1 \ --config configs/sana_video_config/Sana_2000M_720px_ltx2vae_AdamW_fsdp.yaml \ --model_path hf://Efficient-Large-Model/SANA-Video_2B_720p/checkpoints/SANA_Video_2B_720p.pth \ --txt_fileasset/samples/video_prompts_samples.txt \ --cfg_scale 6 \ --motion_score 30 \ --flow_shift 8 \ --work_dir output/sana_t2v_720p_results # 图生视频720p bash inference_video_scripts/inference_sana_video.sh \ --np 1 \ --config configs/sana_video_config/Sana_2000M_720px_ltx2vae_AdamW_fsdp.yaml \ --model_path hf://Efficient-Large-Model/SANA-Video_2B_720p/checkpoints/SANA_Video_2B_720p.pth \ --txt_fileasset/samples/sample_i2v.txt \ --taskltx \ --cfg_scale 6 \ --motion_score 30 \ --flow_shift 8 \ --work_dir output/sana_ti2v_720p_results推理脚本源码解析参数与执行流批量推理的核心实现位于 inference_video_scripts/inference_sana_video.py它以SanaInference(SanaVideoConfig)数据类定义全部 CLI 参数L406-L451再通过pyrallis.parse(config_classSanaInference, config_pathargs.config)将命令行参数与 YAML 配置合并。完整参数表参数默认值说明--configconfigs/sana_video_config/Sana_2000M_480px_AdamW_fsdp.yaml配置文件路径--model_pathhf://.../SANA_Video_2B_480p.pth权重路径支持hf://协议与本地.binFSDP权重--work_dir由 model_path 推导输出根目录视频保存于work_dir/vis/...--txt_file/--json_fileasset/samples/video_prompts_samples.txt/None提示词来源逐行 txt 或 JSON 字典--fps16输出视频帧率--bs1batch size--num_frames-1-1 时使用配置data.num_frames默认 81 帧--cfg_scale6.0Classifier-Free Guidance 强度--flow_shiftNone为空时回落到配置scheduler.inference_flow_shift480p 为 7.0720p 为 8.0--sampling_algoNone为空时使用配置scheduler.vis_sampler默认flow_dpm-solver--skip_typetime_uniform_flow时间步采样方式用于 DPM-Solver--guidance_typeclassifier-free可选adaptive_projected_guidance、classifier-free_STG--seed0随机种子按seed rank分配--step-1采样步数-1 时默认 50 步--motion_score10运动强度见下方 motion prompt 规则--negative_prompt反伪影长句默认即为文档中那段 chaotic sequence... 负面提示词--interval_k0.0图生视频首帧条件的强度--image_split_tokenimage图生视频 txt 中分隔文本与图片路径的 token--prompt_split_tokensplit单行多 prompt 的分隔 token--stg_applied_layers/--stg_scale[]/ 0.0时空引导STG相关--apg_modehw自适应投影引导APG模式--num_cached_blocks-1LongLive 长视频 KV 缓存块数motion prompt 拼接规则源码 L497-L504 定义了运动提示词的生成逻辑motion_score 0在 prompt 末尾追加 motion score: N.如motion score: 30.motion_score 0不追加任何运动提示motion_score 0根据--high_motion追加 high motion或 low motion使用 FastVideoDMD 4 步采样时仅支持high motion。采样器分支visualize函数根据sampling_algo分发到不同 solverL302-L379flow_euler标准 Flow Euler支持 APG 引导flow_dpm-solverDPM-Solver 二阶多步配置文件默认vis_samplerflow_euler_ltx图生视频专用--taskltx/ti2v时自动切换将首帧图像编码进潜空间作为条件帧longlive_flow_euler长视频num_frames超过模型基础帧数时必须启用要求--cfg_scale 1.0结合--num_cached_blocks使用fastvideo_dmd_4step4 步快速采样仅支持规范 T2V可配合--generator_sigma_profile与--high_motion。图生视频的底层实现当config.task ltx时脚本会用vae_encode将输入图像编码为 latent写入噪声张量第 0 帧condition_frame_info {0: 0.0}其余帧继续去噪从而形成首帧条件L264-L300。若配置了图像编码器image_encoder还会额外注入图像 embedding 参与跨模态条件。配置解析480p 与 720p 的关键差异对比两份推理配置可以清楚看到两个分辨率版本在架构上的分工维度480pWanVAE720pLTX2VAE模型SanaMSVideo_2000M_P2_D20patch_size2SanaMSVideo_2000M_P1_D20patch_size1VAEWanVAE16 通道8× 空间下采样stride[4,8,8]LTX2VAE_diffusersAutoencoderKLLTX2Video128 通道32× 空间压缩stride[8,32,32]VAE 精度float32bfloat16宽高比表ASPECT_RATIO_VIDEO_480_MSASPECT_RATIO_VIDEO_720_MS_DIV32保证空间维度被 32 整除文本编码器gemma-2-2b-itmodel_max_length300带 CHI 提示词增强同左flow_shift / inference_flow_shift3.0 / 7.03.0 / 8.0720p 之所以改用 patch_size1是因为 LTX2 VAE 已将空间压缩到 32×patch 不能再叠加两套配置的scheduler均采用linear_flow噪声调度、predict_flow_v: true并使用logit_normal时间步加权logit_mean0.0, logit_std1.0。这些设置在训练与推理间保持一致详见 Sana_2000M_720px_ltx2vae_AdamW_fsdp.yaml 与 docs/sana_video.md 的训练章节。Sana Video LTX2 Refiner 两阶段精修管线这是本仓库视频推理最核心的进阶玩法官方称之为Two-Stage Inference ParadigmBet Small to Win BigStage 1轻量 2B 的 SANA-Video 负责生成时间维度上的结构与运动输出 latent不直接解码Stage 1.5可选的 latent 空间上采样LTX2LatentUpsamplerModel/ patch upsampleStage 2步蒸馏的 LTX-2 Refiner仅 3 步负责空间分辨率与纹理精修以 720p 的延迟获得接近 2K 的画质。仓库中的命令行实现为 app/sana_video_refiner_pipeline_diffusers.pypython app/sana_video_refiner_pipeline_diffusers.py \ --sana_model_id Efficient-Large-Model/SANA-Video_2B_720p_diffusers \ --ltx2_model_id Lightricks/LTX-2 \ --prompt A cat and a dog baking a cake together in a kitchen. \ --sana_height 704 \ --sana_width 1280 \ --sana_frames 81 \ --output_path sana_ltx2_refined.mp4CLI 参数表参数默认值说明--prompt猫狗烘焙示例句生成提示词内部会追加motion score: N.--negative_prompt反伪影长句与原生推理脚本相同的默认负面提示词--sana_model_idEfficient-Large-Model/SANA-Video_2B_720p_diffusersStage-1 模型diffusers 格式--ltx2_model_idLightricks/LTX-2Stage-2 Refiner 基座--sana_height / --sana_width / --sana_frames704 / 1280 / 81Stage-1 生成尺寸与帧数--motion_score30运动强度--sana_guidance_scale / --sana_num_steps6.0 / 50Stage-1 的 CFG 与步数--frame_rate16.0输出帧率--seed42随机种子--output_pathsana_ltx2_refined.mp4Stage-2 最终输出--save_sana_outputsana_original.mp4同时保存 Stage-1 原始输出用于对比置空则跳过--skip_audioTrue是否跳过音频生成--skip_upsamplerTrue是否跳过 latent 上采样False 时使用latent_upsampler完整 Python 参考实现原文档给出了不依赖 CLI、完全可控的 diffusers 级参考代码完整继承如下Sana Video LTX2 Refiner: Stage 1 generate latent → Stage 2 refine (3 steps). import gc import torch from diffusers import SanaVideoPipeline, FlowMatchEulerDiscreteScheduler from diffusers.pipelines.ltx2 import LTX2Pipeline, LTX2LatentUpsamplePipeline from diffusers.pipelines.ltx2.latent_upsampler import LTX2LatentUpsamplerModel from diffusers.pipelines.ltx2.utils import STAGE_2_DISTILLED_SIGMA_VALUES from diffusers.pipelines.ltx2.export_utils import encode_video device cuda dtype torch.bfloat16 prompt A cat walking on the grass, facing the camera. negative_prompt A chaotic sequence with misshapen, deformed limbs in heavy motion blur, sudden disappearance, jump cuts, jerky movements, rapid shot changes, frames out of sync, inconsistent character shapes, temporal artifacts, jitter, and ghosting effects, creating a disorienting visual experience. motion_score 30 height, width, frames, frame_rate 704, 1280, 81, 16.0 seed 42 # ── Load all models ── sana_pipe SanaVideoPipeline.from_pretrained( Efficient-Large-Model/SANA-Video_2B_720p_diffusers, torch_dtypedtype, ) sana_pipe.text_encoder.to(dtype) sana_pipe.enable_model_cpu_offload() ltx_pipe LTX2Pipeline.from_pretrained(Lightricks/LTX-2, torch_dtypedtype) ltx_pipe.load_lora_weights( Lightricks/LTX-2, adapter_namestage_2_distilled, weight_nameltx-2-19b-distilled-lora-384.safetensors, ) ltx_pipe.set_adapters(stage_2_distilled, 1.0) ltx_pipe.vae.to(dtype) ltx_pipe.enable_model_cpu_offload() upsampler_model LTX2LatentUpsamplerModel.from_pretrained( Lightricks/ltx-2-patch-upsampler, torch_dtypedtype, ) upsampler LTX2LatentUpsamplePipeline(upsampler_model).to(device) # ── Stage 1: Sana-Video generates latent ── motion_prompt f motion score: {motion_score}. sana_out sana_pipe( promptprompt motion_prompt, negative_promptnegative_prompt, heightheight, widthwidth, framesframes, guidance_scale6, num_inference_steps50, generatortorch.Generator(devicedevice).manual_seed(seed), output_typelatent, ) sana_latent sana_out.frames del sana_pipe; gc.collect(); torch.cuda.empty_cache() # ── Stage 1.5: Upsample latent ── video_latent sana_latent.squeeze(0).permute(1, 0, 2, 3) packed upsampler(video_latent, output_typelatent).frames lF, _, lH, lW packed.shape pH lH * ltx_pipe.vae_spatial_compression_ratio pW lW * ltx_pipe.vae_spatial_compression_ratio pT (lF - 1) * ltx_pipe.vae_temporal_compression_ratio 1 dur pT / frame_rate audio_frames round(dur * ltx_pipe.audio_sampling_rate / ltx_pipe.audio_hop_length / ltx_pipe.audio_vae_temporal_compression_ratio) nch ltx_pipe.audio_vae.config.latent_channels mel ltx_pipe.audio_vae.config.mel_bins // ltx_pipe.audio_vae_mel_compression_ratio audio_latent ( ltx_pipe.audio_vae.latents_mean.unsqueeze(0).unsqueeze(0) .expand(1, audio_frames, nch * mel).to(dtypedtype, devicedevice).contiguous() .unflatten(2, (nch, mel)).permute(0, 2, 1, 3).contiguous() ) del video_latent; gc.collect(); torch.cuda.empty_cache() # ── Stage 2: LTX2 refine ── video, _ ltx_pipe( latentspacked, audio_latentsaudio_latent, promptprompt, negative_promptnegative_prompt, heightpH, widthpW, num_framespT, num_inference_steps3, noise_scaleSTAGE_2_DISTILLED_SIGMA_VALUES[0], sigmasSTAGE_2_DISTILLED_SIGMA_VALUES, guidance_scale1.0, frame_rateframe_rate, generatortorch.Generator(devicedevice).manual_seed(seed), output_typenp, return_dictFalse, ) video torch.from_numpy((video * 255).round().astype(uint8)) encode_video(video[0], fpsframe_rate, audioNone, audio_sample_rateNone, output_pathsana_ltx2_refined.mp4)四个关键技术点结合 app/sana_video_refiner_pipeline_diffusers.py 的源码注释这条管线有四个必须理解的细节Stage-1 输出 latent 而非像素SanaVideoPipeline以output_typelatent运行通过遍历输出对象的latents/video_latents/frames等字段提取 latent源码 L74-L87随后释放 Sana 模型并清理显存手动 pack latent 以跳过 diffusers 的 normalize源码使用LTX2Pipeline._pack_latents(...)手动完成 patch 打包patch_size 与 patch_size_t 取自 transformer 配置保证与官方 LTX-2 原版代码数值一致构造归一化后为零的音频 latent直接用audio_vae.latents_mean平铺构造音频 latent这样经过内部 normalize 后音频通道恰好为零等价于无音频避免生成管线因缺音频输入而报错源码 L180-L196Stage-2 使用蒸馏 3 步 sigma 调度STAGE_2_DISTILLED_SIGMA_VALUES (0.909375, 0.725, 0.421875, 0.0)num_inference_steps3、guidance_scale1.0配合stage_2_distilledLoRAltx-2-19b-distilled-lora-384.safetensors做确定性 Euler 去噪。另外仓库 diffusion/refiner/diffusers_ltx2_refiner.py 中还提供了同一套 sigma 常量L47以及面向 Sana-WM 的分块自回归精修实现block_size3、kv_max_frames11、sink_size1的滑窗 KV 缓存可作为将本管线扩展到流式/长视频场景的参考。模型转换pth → diffusers safetensors原生.pth检查点可以通过仓库转换脚本导出为 diffusers 格式以便配合SanaVideoPipeline/SanaImageToVideoPipeline使用python tools/convert_scripts/convert_sana_video_to_diffusers.py --dump_path output/SANA_Video_2B_480p_diffusers --save_full_pipeline转换脚本位于 tools/convert_scripts/convert_sana_video_to_diffusers.py--save_full_pipeline会连同 VAE、文本编码器与 scheduler 一起保存为完整 pipeline 目录。该步骤是使用 diffusers 版两阶段管线上一节的必要前置条件之一。与 LongSANA 等扩展的衔接如果你需要生成长视频分钟级原生推理脚本还集成了longlive_flow_euler采样器当--num_frames超过模型基础帧数配置data.num_frames默认 81时脚本会断言必须使用该采样器并通过--num_cached_blocks控制 KV 缓存的保留块数见 inference_video_scripts/inference_sana_video.py。官方 LongSANASANA-Video LongLive的相关性能陈述与说明见 docs/longsana.md 与 docs/sana_video.md。参考与延伸阅读docs/sana_video.mdSANA-Video 官方主文档含 diffusersSanaVideoPipeline/SanaImageToVideoPipeline用法、训练与 VBench 性能数据docs/model_zoo.md480p / 720p 模型权重清单inference_video_scripts/inference_sana_video.py批量推理脚本源码app/sana_video_refiner_pipeline_diffusers.py两阶段精修 CLI 实现diffusion/refiner/diffusers_ltx2_refiner.pyLTX-2 Refiner 的分块自回归实现asset/samples/video_prompts_samples.txt 与 asset/samples/sample_i2v.txt文生视频 / 图生视频提示词样本tests/test_sana_video2.pySanaVideo2 相关测试用例tools/convert_scripts/convert_sana_video_to_diffusers.py.pth权重转 diffusers 格式使用前提与限制说明本文所有命令均以当前仓库快照为基准配置文件路径以configs/sana_video_config/目录下的实际文件名为准app/目录中当前未包含文档早期版本的sana_video_pipeline.py若需单条推理可改用inference_video_scripts/inference_sana_video.py或 diffusers 管线。两阶段精修依赖的 diffusers 需为包含SanaVideoPipeline与 LTX-2 组件的较新版本请在安装依赖时以 docs/requirements.txt 与实际运行报错为准。【免费下载链接】SanaSANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer项目地址: https://gitcode.com/GitHub_Trending/sana/Sana创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考