ikddlmZmZddlZddlmZddlmZddlm Z m Z ddl m Z ddl mZd d lmZmZmZmZd d lmZmZmZd d lmZej2eZGddeZGddeZGddeZGddeZGddeZ GddeZ!GddeZ"GddeZ#Gdde#Z$Gd d!e#Z%Gd"d#e#Z&Gd$d%e#Z'Gd&d'e#Z(Gd(d)e#Z)y)*)ListTupleN) FrozenDict)ClassifierFreeGuidance)QwenImageControlNetModelQwenImageTransformer2DModel)FlowMatchEulerDiscreteScheduler)logging) BlockStateLoopSequentialPipelineBlocksModularPipelineBlocks PipelineState) ComponentSpec InputParam OutputParam)QwenImageModularPipelinec eZdZdZedefdZedeefdZ e jde de dede jfd Zy ) QwenImageLoopBeforeDenoiser qwenimagereturnc yNzstep within the denoising loop that prepares the latent input for the denoiser. This block should be used to compose the `sub_blocks` attribute of a `LoopSequentialPipelineBlocks` object (e.g. `QwenImageDenoiseLoopWrapper`)selfs w/home/obispo/Crisostomo_bridge/mision_env/lib/python3.12/site-packages/diffusers/modular_pipelines/qwenimage/denoise.py descriptionz'QwenImageLoopBeforeDenoiser.description#  : c>tddtjdgS)NlatentsT^The initial latents to use for the denoising process. Can be generated in prepare_latent step.required type_hintr rtorchTensorrs rinputsz"QwenImageLoopBeforeDenoiser.inputs+s& ,,|    r" components block_stateitc|j|jjdj|jj|_|j|_||fS)Nr)expandr$shapetodtypetimesteplatent_model_inputrr-r.r/r0s r__call__z$QwenImageLoopBeforeDenoiser.__call__6sV !xx (;(;(A(A!(DEHHI\I\IbIbc )4)<)< &;&&r"N__name__ __module__ __qualname__ model_namepropertystrr rrr,r*no_gradrr intr+r9rr"rrr sJ  S   Z(  U]]_'#;'*'Y\'afamam''r"rc eZdZdZedefdZedeefdZ e jde de dede jfd Zy ) QwenImageEditLoopBeforeDenoiserrrc yrrrs rr z+QwenImageEditLoopBeforeDenoiser.descriptionAr!r"cvtddtjdtddtjdgS)Nr$Tr%r& image_latentszThe initial image latents to use for the denoising process. Can be encoded in vae_encoder step and packed in prepare_image_latents step.r)rs rr,z&QwenImageEditLoopBeforeDenoiser.inputsIsC ,,|   ,,g    r"r-r.r/r0ctj|j|jgd|_|j |jj dj|jj|_ ||fS)Nr)dimr) r*catr$rGr7r2r3r4r5r6r8s rr9z(QwenImageEditLoopBeforeDenoiser.__call__Zsm*/K4G4GIbIb3cij)k & xx (;(;(A(A!(DEHHI\I\IbIbc ;&&r"Nr:rr"rrDrD>sJ  S   Z(  U]]_'#;'*'Y\'afamam''r"rDc eZdZdZedeefdZedefdZ edee fdZ e jdededed efd Zy ) %QwenImageLoopBeforeDenoiserControlNetrrc^tdttddidtdtgS)Nguiderguidance_scale@ from_configconfigdefault_creation_method controlnet)rrrrrs rexpected_componentsz9QwenImageLoopBeforeDenoiserControlNet.expected_componentsfs: &!#3S"9:(5   ,(@ A  r"c y)Nzstep within the denoising loop that runs the controlnet before the denoiser. This block should be used to compose the `sub_blocks` attribute of a `LoopSequentialPipelineBlocks` object (e.g. `QwenImageDenoiseLoopWrapper`)rrs rr z1QwenImageLoopBeforeDenoiserControlNet.descriptionrr!r"c tddtjdtdtdtddttd td dt d td d gS)Ncontrol_image_latentsTzgThe control image to use for the denoising process. Can be generated in prepare_controlnet_inputs step.r&controlnet_conditioning_scalez}The controlnet conditioning scale value to use for the denoising process. Can be generated in prepare_controlnet_inputs step.r(r controlnet_keepzpThe controlnet keep values to use for the denoising process. Can be generated in prepare_controlnet_inputs step.num_inference_stepsgThe number of inference steps to use for the denoising process. Can be generated in set_timesteps step.denoiser_input_fieldszAll conditional model inputs for the denoiser. It should contain prompt_embeds/negative_prompt_embeds, txt_seq_lens/negative_txt_seq_lens. kwargs_typer )rr*r+floatrrBrs rr,z,QwenImageLoopBeforeDenoiserControlNet.inputszs ',,F   /\  !u+O   %F   3r 1  r"r-r.r/r0c 6t|j|tr?t|j|j|Dcgc] \}}||z c}}|_n8|j}t|tr|d}||j|z|_|j |j|j|j |jdz |j|j|j|jd }||jd<||fScc}}w)NrF) hidden_statescontrolnet_condconditioning_scaler6 img_shapesencoder_hidden_statesencoder_hidden_states_mask txt_seq_lens return_dictcontrolnet_block_samples) isinstancer\listziprZ cond_scalerUr7rYr6rh prompt_embedsprompt_embeds_maskrkadditional_cond_kwargs) rr-r.r/r0cscontrolnet_cond_scalerms rr9z.QwenImageLoopBeforeDenoiserControlNet.__call__s k11!4d ;"%k&O&OQ\QlQlmnQo"p&!QA&K "%0$M$M !/6(=a(@%%:[=X=XYZ=[%[K "$.#8#8%88'==*55 ))D0"--"-";";'2'E'E$11$9 $  Jb **+EF;&&1&sDN)r;r<r=r>r?rrrVr@r rr,r*rArr rBr9rr"rrLrLcsJ   T-%8     S    Z(    DU]]_'#;'*'Y\'ad''r"rLc eZdZdZedefdZedeefdZ edee fdZ e jdededed e j"fd Zy ) QwenImageLoopDenoiserrrc yNzstep within the denoising loop that denoise the latent input for the denoiser. This block should be used to compose the `sub_blocks` attribute of a `LoopSequentialPipelineBlocks` object (e.g. `QwenImageDenoiseLoopWrapper`)rrs rr z!QwenImageLoopDenoiser.descriptionr!r"c^tdttddidtdtgSNrNrOrPrQrR transformerrrrr rs rrVz)QwenImageLoopDenoiser.expected_components: &!#3S"9:(5   -)D E  r"c tdtddtjdtddtdtdd td dtt ttfd gS Nattention_kwargsr$TzWThe latents to use for the denoising process. Can be generated in prepare_latents step.r&r]r^r_z[conditional model inputs for the denoiser: e.g. prompt_embeds, negative_prompt_embeds, etc.r`rhzhThe shape of the image latents for RoPE calculation. Can be generated in prepare_additional_inputs step.rr*r+rBrrrs rr,zQwenImageLoopDenoiser.inputs~ ) * ,,u   %F   3y  uS#X/G  %  r"r-r.r/r0c t|ddt|ddft|ddt|ddft|ddt|ddfd}|jj||j||jj |}|D]}|jj |j |jDcic]}|t||} }|j d|j|jd z |j|jd d | |jd |_ |jj|j |j|} tj | j"d d} tj | j$d d} | j$| | z z|_ ||fScc}w)Nrrnegative_prompt_embedsrsnegative_prompt_embeds_maskrknegative_txt_seq_lensrirjrkstepr]r6rdFrer6rhrrlrTrIkeepdimr)getattrrN set_stater]prepare_inputsprepare_modelsr~keysr7r6rhrrt noise_predcleanup_modelsr*norm pred_condpred) rr-r.r/r0 guider_inputs guider_stateguider_state_batch input_name cond_kwargs guider_outputpred_cond_norm pred_norms rr9zQwenImageLoopDenoiser.__call__s _d; %=tD&  %94@ %BDI+  ^T: %r?r@r rrrVrr,r*rArr rBr+r9rr"rryrysJ  S    T-%8     Z(  6U]]_+'#;+'*+'Y\+'afamam+'+'r"ryc eZdZdZedefdZedeefdZ edee fdZ e jdededed e j"fd Zy ) QwenImageEditLoopDenoiserrrc yr{rrs rr z%QwenImageEditLoopDenoiser.description!r!r"c^tdttddidtdtgSr}rrs rrVz-QwenImageEditLoopDenoiser.expected_components)rr"c tdtddtjdtddtdtdd td dtt ttfd gSrrrs rr,z QwenImageEditLoopDenoiser.inputs5rr"r-r.r/r0c *t|ddt|ddft|ddt|ddft|ddt|ddfd}|jj||j||jj |}|D]}|jj |j |jDcic]}|t||} }|j d|j|jd z |j|jd d | |jd |_ |jj|j |j|} | jddd|j j#d f} | j$ddd|j j#d f} t'j(| dd} t'j(| dd}| | |z z|_ ||fScc}w)NrrrrsrrkrrrrdFrrrrTrr)rrNrr]rrr~rr7r6rhrrtrrrr$sizerr*r)rr-r.r/r0rrrrrrrrrrs rr9z"QwenImageEditLoopDenoiser.__call__Qs7 _d; %=tD&  %94@ %BDI+  ^T: %r?r@r rrrVrrr*rArr rBr+r9rr"rrrsJ  S   T-%8   d;&7  U]]_'#;'*'Y\'afamam''r"rc eZdZdZedefdZedeefdZ e jde de dede jfd Zy ) !QwenImageLoopAfterDenoiserInpaintrrc y)Nzstep within the denoising loop that updates the latents using mask and image_latents for inpainting. This block should be used to compose the `sub_blocks` attribute of a `LoopSequentialPipelineBlocks` object (e.g. `QwenImageDenoiseLoopWrapper`)rrs rr z-QwenImageLoopAfterDenoiserInpaint.descriptionr!r"c tddtjdtddtjdtddtjdtd dtjd gS) NmaskTz]The mask to use for the inpainting process. Can be generated in inpaint prepare latents step.r&rGzfThe image latents to use for the inpainting process. Can be generated in inpaint prepare latents step. initial_noisezfThe initial noise to use for the inpainting process. Can be generated in inpaint prepare latents step. timestepsWThe timesteps to use for the denoising process. Can be generated in set_timesteps step.r)rs rr,z(QwenImageLoopAfterDenoiserInpaint.inputssz ,,{   ,,E   ,,E   ,,u  '  r"r-r.r/r0c|j|_|t|jdz krk|j|dz|_|j j |jtj|jg|j|_d|jz |jz|j|jzz|_ ||fS)Nr) rGinit_latents_properlenrnoise_timestepr scale_noiser*tensorrrr$r8s rr9z*QwenImageLoopAfterDenoiserInpaint.__call__s*5*C*C ' s;(()A- -)4)>)>q1u)EK &.8.B.B.N.N//{?Y?Y>Z1[]h]v]v/K +     + +,.9.>.>ATAT.TU ;&&r"Nr:rr"rrrsJ  S   Z(  8U]]_ '#; '* 'Y\ 'afamam ' 'r"rceZdZdZedefdZedeefdZ edee fdZ e jdededefdZy ) QwenImageDenoiseLoopWrapperrrc y)NzPipeline block that iteratively denoise the latents over `timesteps`. The specific steps with each iteration can be customized with `sub_blocks` attributesrrs rr z'QwenImageDenoiseLoopWrapper.descriptions  d r"c$tdtgSrrrs rloop_expected_componentsz4QwenImageDenoiseLoopWrapper.loop_expected_componentsrr"cbtddtjdtddtdgS)NrTrr&r]r^)rr*r+rBrs r loop_inputsz'QwenImageDenoiseLoopWrapper.loop_inputss? ,,u   %F    r"r-statecx|j|}tt|j|j|j j zz d|_i|_|j|j5}t|jD]{\}}|j||||\}}|t|jdz k(s3|dz|jkDsL|dz|j j zdk(sl|j} ddd|j||||fS#1swYxYw)Nr)total)r/r0r)get_block_statemaxrrr]rordernum_warmup_stepsrt progress_bar enumerate loop_stepupdateset_block_state)rr-rr.rr/r0s rr9z$QwenImageDenoiseLoopWrapper.__call__s/**51 '*  %% &)H)H:K_K_KeKe)e egh( $.0 *   [%D%D  E *!+"7"78 *1*...[TUYZ.*[' KK112Q66Uk:::AI]I]IcIc?cgh?h '')  * * UK05   * *s=A!D0D0?D00D9N)r;r<r=r>r?r@r rrrrrr*rArrr9rr"rrrsJ  S   $}*=   T*-  U]]_!#;!M!Vc!!r"rc4eZdZeeegZgdZede fdZ y)QwenImageDenoiseStepbefore_denoiserdenoiserafter_denoiserrc y)NagDenoise step that iteratively denoise the latents. Its loop logic is defined in `QwenImageDenoiseLoopWrapper.__call__` method At each iteration, it runs blocks defined in `sub_blocks` sequencially: - `QwenImageLoopBeforeDenoiser` - `QwenImageLoopDenoiser` - `QwenImageLoopAfterDenoiser` This block supports text2image and image2image tasks for QwenImage.rrs rr z QwenImageDenoiseStep.description%s  R r"N) r;r<r=rryr block_classes block_namesr?r@r rr"rrrs3#"M DK   S    r"rc6eZdZeeeegZgdZe de fdZ y)QwenImageInpaintDenoiseSteprrrafter_denoiser_inpaintrc y)Na~Denoise step that iteratively denoise the latents. Its loop logic is defined in `QwenImageDenoiseLoopWrapper.__call__` method At each iteration, it runs blocks defined in `sub_blocks` sequencially: - `QwenImageLoopBeforeDenoiser` - `QwenImageLoopDenoiser` - `QwenImageLoopAfterDenoiser` - `QwenImageLoopAfterDenoiserInpaint` This block supports inpainting tasks for QwenImage.rrs rr z'QwenImageInpaintDenoiseStep.description<s  B r"N) r;r<r=rryrrrrr?r@r rr"rrr3s6#") M ^K   S    r"rc6eZdZeeeegZgdZe de fdZ y)QwenImageControlNetDenoiseStep)rbefore_denoiser_controlnetrrrc y)NaDenoise step that iteratively denoise the latents. Its loop logic is defined in `QwenImageDenoiseLoopWrapper.__call__` method At each iteration, it runs blocks defined in `sub_blocks` sequencially: - `QwenImageLoopBeforeDenoiser` - `QwenImageLoopBeforeDenoiserControlNet` - `QwenImageLoopDenoiser` - `QwenImageLoopAfterDenoiser` This block supports text2img/img2img tasks with controlnet for QwenImage.rrs rr z*QwenImageControlNetDenoiseStep.descriptionTs  X r"N) r;r<r=rrLryrrrr?r@r rr"rrrKs6#-" M bK   S    r"rc8eZdZeeeeegZgdZ e de fdZ y)%QwenImageInpaintControlNetDenoiseStep)rrrrrrc y)NaDenoise step that iteratively denoise the latents. Its loop logic is defined in `QwenImageDenoiseLoopWrapper.__call__` method At each iteration, it runs blocks defined in `sub_blocks` sequencially: - `QwenImageLoopBeforeDenoiser` - `QwenImageLoopBeforeDenoiserControlNet` - `QwenImageLoopDenoiser` - `QwenImageLoopAfterDenoiser` - `QwenImageLoopAfterDenoiserInpaint` This block supports inpainting tasks with controlnet for QwenImage.rrs rr z1QwenImageInpaintControlNetDenoiseStep.descriptionss  R r"N) r;r<r=rrLryrrrrr?r@r rr"rrrcs:#-") MK  S    r"rc4eZdZeeegZgdZede fdZ y)QwenImageEditDenoiseSteprrc y)NaODenoise step that iteratively denoise the latents. Its loop logic is defined in `QwenImageDenoiseLoopWrapper.__call__` method At each iteration, it runs blocks defined in `sub_blocks` sequencially: - `QwenImageEditLoopBeforeDenoiser` - `QwenImageEditLoopDenoiser` - `QwenImageLoopAfterDenoiser` This block supports QwenImage Edit.rrs rr z$QwenImageEditDenoiseStep.descriptions  2 r"N) r;r<r=rDrrrrr?r@r rr"rrrs3'!"M DK   S    r"rc6eZdZeeeegZgdZe de fdZ y)QwenImageEditInpaintDenoiseSteprrc y)NaDenoise step that iteratively denoise the latents. Its loop logic is defined in `QwenImageDenoiseLoopWrapper.__call__` method At each iteration, it runs blocks defined in `sub_blocks` sequencially: - `QwenImageEditLoopBeforeDenoiser` - `QwenImageEditLoopDenoiser` - `QwenImageLoopAfterDenoiser` - `QwenImageLoopAfterDenoiserInpaint` This block supports inpainting tasks for QwenImage Edit.rrs rr z+QwenImageEditInpaintDenoiseStep.descriptions  G r"N) r;r<r=rDrrrrrr?r@r rr"rrrs6'!") M ^K   S    r"r)*typingrrr*configuration_utilsrguidersrmodelsrr schedulersr utilsr modular_pipeliner rrrmodular_pipeline_utilsrrrr get_loggerr;loggerrrDrLryrrrrrrrrrrrr"rrs --K9mmKK6   H %'"7'<"'&;"'JV',AV'r_'1_'Db' 5b'J&'!6&'R5'(=5'p5!">5!r 6 , "= 0 %@ 0 ,G @ : * &A r"