i<ddlmZddlmZmZmZddlZddlmcm Z ddlmZddl m Z m Z ddlmZddlmZdd lmZdd lmZd d lmZd d lmZmZmZmZd dlmZd dlm Z ddl!m"Z"m#Z#m$Z$m%Z%eGddeZ&Gddeee"e Z'y)) dataclass)OptionalTupleUnionN)nn) ConfigMixinregister_to_config)ConsistencyDecoderScheduler) BaseOutput)apply_forward_hook) randn_tensor)AttentionMixin)ADDED_KV_ATTENTION_PROCESSORSCROSS_ATTENTION_PROCESSORSAttnAddedKVProcessor AttnProcessor) ModelMixin) UNet2DModel)AutoencoderMixin DecoderOutputDiagonalGaussianDistributionEncoderceZdZUdZded<y)ConsistencyDecoderVAEOutputa2 Output of encoding method. Args: latent_dist (`DiagonalGaussianDistribution`): Encoded outputs of `Encoder` represented as the mean and logvar of `DiagonalGaussianDistribution`. `DiagonalGaussianDistribution` allows for sampling latents from the distribution. r latent_distN)__name__ __module__ __qualname____doc____annotations__/home/obispo/Crisostomo_bridge/mision_env/lib/python3.12/site-packages/diffusers/models/autoencoders/consistency_decoder_vae.pyrr&s0/r%rc4eZdZdZdZe d/dedededede edfd e d e edfd ed ed edede de edfde edfdededededededededede edff0fd Z dZ e d0dejde d eee effd!Ze d1d"ejd#eej*de d$ed eee ejff d%Zd&ejd'ejd(ed ejfd)Zd&ejd'ejd(ed ejfd*Zd0dejde d eee ffd+Z d2d,ejd-e de d#eej*d eee ejff d.ZxZS)3ConsistencyDecoderVAEaP The consistency decoder used with DALL-E 3. Examples: ```py >>> import torch >>> from diffusers import StableDiffusionPipeline, ConsistencyDecoderVAE >>> vae = ConsistencyDecoderVAE.from_pretrained("openai/consistency-decoder", torch_dtype=torch.float16) >>> pipe = StableDiffusionPipeline.from_pretrained( ... "stable-diffusion-v1-5/stable-diffusion-v1-5", vae=vae, torch_dtype=torch.float16 ... ).to("cuda") >>> image = pipe("horse", generator=torch.manual_seed(0)).images[0] >>> image ``` Fscaling_factorlatent_channels sample_sizeencoder_act_fnencoder_block_out_channels.encoder_double_zencoder_down_block_typesencoder_in_channelsencoder_layers_per_blockencoder_norm_num_groupsencoder_out_channelsdecoder_add_attentiondecoder_block_out_channelsdecoder_down_block_typesdecoder_downsample_paddingdecoder_in_channelsdecoder_layers_per_blockdecoder_norm_epsdecoder_norm_num_groupsdecoder_num_train_timestepsdecoder_out_channelsdecoder_resnet_time_scale_shiftdecoder_time_embedding_typedecoder_up_block_typesc|t|t|||||| | | |_t | | ||||||||||| |_t |_|j||jd|jdtjgddddddfd|jd tjgd dddddfdtjd |zd |zd |_d|_d|_|j"j$|_t)|j"j$t*t,fr|j"j$d n|j"j$}t/|d t1|j"j2d z zz |_d|_y)N)act_fnblock_out_channelsdouble_zdown_block_types in_channelslayers_per_blocknorm_num_groups out_channels) add_attentionrCrEdownsample_paddingrFrGnorm_epsrHnum_train_timestepsrIresnet_time_scale_shifttime_embedding_typeup_block_types)rCF) force_upcastmeans)gg:?gyD?glL?g N3^) persistentstds)g4?gn=?gr ^?gr` ?rrrg?)super__init__rencoderr decoder_unetr decoder_schedulerr register_buffertorchtensorrConv2d quant_conv use_slicing use_tilingconfigr+tile_sample_min_size isinstancelisttupleintlenrCtile_latent_min_sizetile_overlap_factor)selfr)r*r+r,r-r.r/r0r1r2r3r4r5r6r7r8r9r:r;r<r=r>r?r@ __class__s r&rVzConsistencyDecoderVAE.__init__IsV !9%5+53-   (/959+5%3 ;-$C ;1 "=!> 3MN U3   LLI J4QRTXZ^K^ _   ELL!OPQUWXZ^`dQderw  ))A$7_9LaP %)KK$;$;!$++11D%=A KK # #A &((  %( qSA_A_=`cd=d7e(f$g!#' r%c jtd|jjDr t}nmtd|jjDr t }n8t dt t|jj|j|y)ze Disables custom attention processors and sets the default attention implementation. c3@K|]}|jtvywN)rkr.0procs r& zCConsistencyDecoderVAE.set_default_attn_processor..si4t~~!>>ic3@K|]}|jtvywrn)rkrros r&rrzCConsistencyDecoderVAE.set_default_attn_processor..sh$#==hrszOCannot call `set_default_attn_processor` when attention processors are of type N) allattn_processorsvaluesrr ValueErrornextiterset_attn_processor)rj processors r&set_default_attn_processorz0ConsistencyDecoderVAE.set_default_attn_processors i4K_K_KfKfKhi i,.I h$J^J^JeJeJgh h%IabfgklpmAmAmHmHmJhKcLbMN   *r%x return_dictreturnc|jrK|jd|jkDs|jd|jkDr|j||S|jrU|jddkDrC|j dDcgc]}|j |}}tj|}n|j |}|j|}t|}|s|fSt|Scc}w)al Encode a batch of images into latents. Args: x (`torch.Tensor`): Input batch of images. return_dict (`bool`, *optional*, defaults to `True`): Whether to return a [`~models.autoencoders.consistency_decoder_vae.ConsistencyDecoderVAEOutput`] instead of a plain tuple. Returns: The latent representations of the encoded images. If `return_dict` is True, a [`~models.autoencoders.consistency_decoder_vae.ConsistencyDecoderVAEOutput`] is returned, otherwise a plain `tuple` is returned. )rrrr) r`shaperb tiled_encoder_splitrWr[catr^rr)rjr~rx_sliceencoded_sliceshmoments posteriors r&encodezConsistencyDecoderVAE.encodes$ ?? d.G.G G177SU;Y]YrYrKr$$QK$@ @    QCD771:Ndll73NNN .)A QA//!$09 < *yAAOs C7z generatornum_inference_stepsc:||jjz|jz |jz }dt |jj dz z}t j|d|}|j\}}}} |jj||j|jjt|d|| f||j|jz} |jjD]} t!j"|jj%| | |gd} |j'| | j(d d d dd d d d f} |jj+| | | |j,}|} | }|s|fSt/| S) a Decodes the input latent vector `z` using the consistency decoder VAE model. Args: z (torch.Tensor): The input latent vector. generator (Optional[torch.Generator]): The random number generator. Default is None. return_dict (bool): Whether to return the output as a dictionary. Default is True. num_inference_steps (int): The number of inference steps. Default is 2. Returns: Union[DecoderOutput, Tuple[torch.Tensor]]: The decoded output. rrnearest)mode scale_factor)devicer)rdtyperdimNsample)rar)rRrTrgrCF interpolaterrY set_timestepsrinit_noise_sigmarr timestepsr[concatscale_model_inputrXrstep prev_sampler)rjrrrrr batch_size_heightwidthx_tt model_input model_outputrx_0s r&decodezConsistencyDecoderVAE.decodesy*++ +djj 8DII ES!?!?@1DE MM!), G'(ww$ Avu ,,-@,U$$55 FE *iqwwWXW_W_9  ''11 A,,(>(>(P(PQTVW(XZ['\bcdK,,[!<CCArr1aKPL0055lAsIVbbKC   6MC((r%ab blend_extentct|jd|jd|}t|D]A}|dddd| |zddfd||z z z|dddd|ddf||z zz|dddd|ddf<C|S)Nrrminrrange)rjrrrys r&blend_vzConsistencyDecoderVAE.blend_v s1771:qwwqz<@ |$ xAa\MA$5q89Q\AQ=QRUVWXZ[]^`aWaUbfgjvfvUwwAaAqjM xr%ct|jd|jd|}t|D]A}|dddddd| |zfd||z z z|dddddd|f||z zz|dddddd|f<C|S)Nrrr)rjrrrr~s r&blend_hzConsistencyDecoderVAE.blend_hs1771:qwwqz<@ |$ xAaA }q'889Q\AQ=QRUVWXZ[]^`aWaUbfgjvfvUwwAaAqjM xr%c t|jd|jz z}t|j|jz}|j|z }g}t d|j d|D]}g}t d|j d|D]`} |dddd|||jz| | |jzf} |j | } |j| } |j| b|j|g} t|D]\}}g} t|D]d\} } |dkDr|j||dz | | |} | dkDr|j|| dz | |} | j| ddddd|d|ff| jtj| dtj| d} t| }|s|fSt|S)aEncode a batch of images using a tiled encoder. When this option is enabled, the VAE will split the input tensor into tiles to compute encoding in several steps. This is useful to keep memory use constant regardless of image size. The end result of tiled encoding is different from non-tiled encoding because each tile uses a different encoder. To avoid tiling artifacts, the tiles overlap and are blended together to form a smooth output. You may still see tile-sized changes in the output, but they should be much less noticeable. Args: x (`torch.Tensor`): Input batch of images. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~models.autoencoders.consistency_decoder_vae.ConsistencyDecoderVAEOutput`] instead of a plain tuple. Returns: [`~models.autoencoders.consistency_decoder_vae.ConsistencyDecoderVAEOutput`] or `tuple`: If return_dict is True, a [`~models.autoencoders.consistency_decoder_vae.ConsistencyDecoderVAEOutput`] is returned, otherwise a plain `tuple` is returned. rrrrNrr)rfrbrirhrrrWr^append enumeraterrr[rrr)rjr~r overlap_sizer row_limitrowsirowjtile result_rows result_rowrrs r&rz"ConsistencyDecoderVAE.tiled_encodes(444D F4q5<<QU AlKDq5<<AE D,GD!!$q!ZiZ)'C"DE F   uyy; < =))KQ/09 < *yAAr%rsample_posteriorc|}|j|j}|r|j|}n|j}|j ||j}|s|fSt |S)a Args: sample (`torch.Tensor`): Input sample. sample_posterior (`bool`, *optional*, defaults to `False`): Whether to sample from the posterior. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`DecoderOutput`] instead of a plain tuple. generator (`torch.Generator`, *optional*, defaults to `None`): Generator to use for sampling. Returns: [`DecoderOutput`] or `tuple`: If return_dict is True, a [`DecoderOutput`] is returned, otherwise a plain `tuple` is returned. )rr)rrrrrr) rjrrrrr~rrdecs r&forwardzConsistencyDecoderVAE.forwardQsk* KKN..   9 5A Akk!yk1886MC((r%)g{P? silu)rT)DownEncoderBlock2DrrrrrrrF)i@ir)ResnetDownsampleBlock2Drrrrrgh㈵>rr scale_shiftlearned)ResnetUpsampleBlock2Drrr)T)NTr)FTN)rr r!r"_supports_group_offloadingr floatrfstrrboolrVr}r r[Tensorrrrrr Generatorrrrrrr __classcell__)rks@r&r(r(4sX$"'!( $6J!%5 $%()')$%&+6L5 +,#$()"'')+/$%/<+43 G^(^(^( ^(  ^( %*#s(O ^(^(#(S/^(!^(#&^( "%!^(""#^($ $%^(&%*#s(O'^((#(S/)^(4%(5^(6!7^(8#&9^(: ;^(<"%=^(>&)?^(@"A^(B*-C^(D&)E^(F!&c3hG^(^(B +37 B B,0 B *E2N,OO P B BD04 #$ ,) <<,)EOO,,) ,) ! ,) }eELL11 2 ,),)^%,,cell%,,cell 5Bell5B5BOjlqOqIr5Bt"' /3 )  ) ) ) EOO, ) }eELL11 2 )r%r()( dataclassesrtypingrrrr[torch.nn.functionalr functionalrconfiguration_utilsr r schedulersr utilsr utils.accelerate_utilsr utils.torch_utilsr attentionrattention_processorrrrrmodeling_utilsr unets.unet_2drvaerrrrrr(r$r%r&rsv")) B58-& ('WW  0* 0  0})J8H+})r%