iȲ ddlZddlmZmZmZmZmZddlZddl Z ddl m Z m Z ddl mZmZddlmZmZmZerddlZ dded ed ed d e j0fd ZGddee Zy)N)ListLiteralOptionalTupleUnion) ConfigMixinregister_to_config) deprecateis_scipy_available)KarrasDiffusionSchedulersSchedulerMixinSchedulerOutputnum_diffusion_timestepsmax_betaalpha_transform_type)cosineexpreturnc $|dk(rd}n|dk(rd}ntd|g}t|D]<}||z }|dz|z }|jtd||||z z |>t j |tj S)aB Create a beta schedule that discretizes the given alpha_t_bar function, which defines the cumulative product of (1-beta) over time from t = [0,1]. Contains a function alpha_bar that takes an argument t and transforms it to the cumulative product of (1-beta) up to that part of the diffusion process. Args: num_diffusion_timesteps (`int`): The number of betas to produce. max_beta (`float`, defaults to `0.999`): The maximum beta to use; use values lower than 1 to avoid numerical instability. alpha_transform_type (`"cosine"` or `"exp"`, defaults to `"cosine"`): The type of noise schedule for `alpha_bar`. Choose from `cosine` or `exp`. Returns: `torch.Tensor`: The betas used by the scheduler to step the model outputs. rcftj|dzdz tjzdz dzS)NgMb?gT㥛 ?r)mathcospits x/home/obispo/Crisostomo_bridge/mision_env/lib/python3.12/site-packages/diffusers/schedulers/scheduling_deis_multistep.py alpha_bar_fnz)betas_for_alpha_bar..alpha_bar_fn<s-88QY%/$''9A=>!C Crc2tj|dzS)Ng()rrrs rrz)betas_for_alpha_bar..alpha_bar_fnAs88AI& &r z"Unsupported alpha_transform_type: r dtype) ValueErrorrangeappendmintorchtensorfloat32)rrrrbetasit1t2s rbetas_for_alpha_barr/"s0x' D  & '=>R=STUU E * +M ( (!e. . S\"- R0@@@(KLM <<U]] 33r c0eZdZdZeDcgc]}|j c}}ZdZe dIde de d e d e d d e e ejee fd e de ddede de de dde ddede ede ede ede ede e de dde dede dd df.d!Zed e e fd"Zed e e fd#ZdJd$e d dfd%Z dKd&e d'e eej2fd(e e d dfd)Zd*ej6d ej6fd+Zd,ejd-ejd ejfd.Zd,ej6d eej6ej6ffd/Zd0ej6d&e d ej6fd1Z d0ej6d&e d ej6fd2Z! dLd0ej6d&e d3e d4e d ej6f d5Z"dd6d7ej6d*ej6d ej6fd8Z#dd6d7ej6d*ej6d ej6fd9Z$dd6d:eej6d*ej6d ej6fd;Z%dd6d:eej6d*ej6d ej6fd<Z& dMd=e e ej6fd>e ej6d e fd?Z'd=e e ej6fd dfd@Z( dNd7ej6d=e e ej6fd*ej6dAed e e)eff dBZ*d*ej6d ej6fdCZ+dDej6dEej6dFejXd ej6fdGZ-d e fdHZ.ycc}}w)ODEISMultistepScheduleru  `DEISMultistepScheduler` is a fast high order solver for diffusion ordinary differential equations (ODEs). This model inherits from [`SchedulerMixin`] and [`ConfigMixin`]. Check the superclass documentation for the generic methods the library implements for all schedulers such as loading and saving. Args: num_train_timesteps (`int`, defaults to `1000`): The number of diffusion steps to train the model. beta_start (`float`, defaults to `0.0001`): The starting `beta` value of inference. beta_end (`float`, defaults to `0.02`): The final `beta` value. beta_schedule (`"linear"`, `"scaled_linear"`, or `"squaredcos_cap_v2"`, defaults to `"linear"`): The beta schedule, a mapping from a beta range to a sequence of betas for stepping the model. Choose from `linear`, `scaled_linear`, or `squaredcos_cap_v2`. trained_betas (`np.ndarray` or `List[float]`, *optional*): Pass an array of betas directly to the constructor to bypass `beta_start` and `beta_end`. solver_order (`int`, defaults to `2`): The DEIS order which can be `1` or `2` or `3`. It is recommended to use `solver_order=2` for guided sampling, and `solver_order=3` for unconditional sampling. prediction_type (`"epsilon"`, `"sample"`, `"v_prediction"`, or `"flow_prediction"`, defaults to `"epsilon"`): Prediction type of the scheduler function; can be `epsilon` (predicts the noise of the diffusion process), `sample` (directly predicts the noisy sample`), `v_prediction` (see section 2.4 of [Imagen Video](https://huggingface.co/papers/2210.02303) paper), or `flow_prediction`. thresholding (`bool`, defaults to `False`): Whether to use the "dynamic thresholding" method. This is unsuitable for latent-space diffusion models such as Stable Diffusion. dynamic_thresholding_ratio (`float`, defaults to `0.995`): The ratio for the dynamic thresholding method. Valid only when `thresholding=True`. sample_max_value (`float`, defaults to `1.0`): The threshold value for dynamic thresholding. Valid only when `thresholding=True`. algorithm_type (`"deis"`, defaults to `"deis"`): The algorithm type for the solver. solver_type (`"logrho"`, defaults to `"logrho"`): Solver type for DEIS. lower_order_final (`bool`, defaults to `True`): Whether to use lower-order solvers in the final steps. Only valid for < 15 inference steps. use_karras_sigmas (`bool`, *optional*, defaults to `False`): Whether to use Karras sigmas for step sizes in the noise schedule during the sampling process. If `True`, the sigmas are determined according to a sequence of noise levels {σi}. use_exponential_sigmas (`bool`, *optional*, defaults to `False`): Whether to use exponential sigmas for step sizes in the noise schedule during the sampling process. use_beta_sigmas (`bool`, *optional*, defaults to `False`): Whether to use beta sigmas for step sizes in the noise schedule during the sampling process. Refer to [Beta Sampling is All You Need](https://huggingface.co/papers/2407.12173) for more information. use_flow_sigmas (`bool`, *optional*, defaults to `False`): Whether to use flow sigmas for step sizes in the noise schedule during the sampling process. flow_shift (`float`, *optional*, defaults to `1.0`): The flow shift parameter for flow-based models. timestep_spacing (`"linspace"`, `"leading"`, or `"trailing"`, defaults to `"linspace"`): The way the timesteps should be scaled. Refer to Table 2 of the [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://huggingface.co/papers/2305.08891) for more information. steps_offset (`int`, defaults to `0`): An offset added to the inference steps, as required by some model families. use_dynamic_shifting (`bool`, defaults to `False`): Whether to use dynamic shifting for the noise schedule. time_shift_type (`"exponential"`, defaults to `"exponential"`): The type of time shifting to apply. r Ndeislogrho exponentialnum_train_timesteps beta_startbeta_end beta_schedule)linear scaled_linearsquaredcos_cap_v2 trained_betas solver_orderprediction_type)epsilonsample v_predictionflow_prediction thresholdingdynamic_thresholding_ratiosample_max_valuealgorithm_type solver_typelower_order_finaluse_karras_sigmasuse_exponential_sigmasuse_beta_sigmasuse_flow_sigmas flow_shifttimestep_spacing)linspaceleadingtrailing steps_offsetuse_dynamic_shiftingtime_shift_typerc|jjrts tdt |jj|jj |jj gdkDr td|+tj|tj|_ n|dk(r-tj|||tj|_ nk|dk(r6tj|dz|dz|tjdz|_ n0|d k(rt||_ nt|d |jd |jz |_tj"|j d |_tj&|j$|_tj&d|j$z |_tj,|j(tj,|j*z |_d|j$z |j$z dz|_d |_| dvr1| dvr|j5dnt| d |j| dvr2| dvr|j5dntd| d |jd|_t9jd |dz |t8jdddj;}tj<||_dg|z|_ d |_!d|_"d|_#|j0jId|_y)Nz:Make sure to install scipy if you want to use beta sigmas.r znOnly one of `config.use_beta_sigmas`, `config.use_exponential_sigmas`, `config.use_karras_sigmas` can be used.r"r9r:?rr;z is not implemented for ?rdim)r2) dpmsolverz dpmsolver++r2)rF)r3)midpointheunbh1bh2r3)rGz solver type cpu)%configrKr ImportErrorsumrJrIr$r(r)r*r+rOr/NotImplementedError __class__alphascumprodalphas_cumprodsqrtalpha_tsigma_tloglambda_tsigmasinit_noise_sigmar num_inference_stepsnpcopy from_numpy timesteps model_outputslower_order_nums _step_index _begin_indexto)selfr5r6r7r8r<r=r>rCrDrErFrGrHrIrJrKrLrMrNrRrSrTrts r__init__zDEISMultistepScheduler.__init__s4 ;; & &/A/CZ[ [ KK//KK66KK11  A   $m5==IDJ h & H>QY^YfYfgDJ o -OcM'--    J1 1,-@ADJ%7OPTP^P^O_&`a aDJJ& #mmDKKQ?zz$"5"56 zz!d&9&9"9:  $,,/%))DLL2II D///43F3FF3N !$  )!==''v'>)^,<;;33 8S8SWd8d dd%'VVBZDKK " ;; ' ': 5 At{{>>BDWZ[D[\2"$!  [[ ) )Y 688=PST=TUJ1&9A&=>KRRTUYWYUYZ[^\^_ddfmmnpnvnvwI 11 1I [[ ) )Z 788;NNJ $++"A"A1zkRXXZ__ahhikiqiqrI NI;;//01JK A 3 33t7J7JJsRSVVF^ ;; ( (WWV_))+F,,vSf,gFSY!Z%$"2"25*"E!Z[aacI^^VVBC[$9:AA"**MF [[ / /WWV_))+F11FXk1lFSY!Z%$"2"25*"E!Z[I^^VVBC[$9:AA"**MF [[ ( (WWV_))+F**VQd*eFSY!Z%$"2"25*"E!Z[I^^VVBC[$9:AA"**MF [[ ( ([[A (G(G$GI\_`I`aF6\FWWT[[33f<T[[E[E[^_E_ciDi@ijklomopuuwF$++"A"AAGGII^^VVBC[$9:AA"**MFYYy"))As6{*CVLFt221559L9LQ9OOTWWJ^^Vj\$:;BB2::NF&&v. )))477vU[[7Y#&y>   KK $ $%!"  kknnU+ I"[ "[ "[s8\>\"6\'r@cb|j}|j^}}}|tjtjfvr|j }|j ||tj|z}|j}tj||jjd}tj|d|jj}|jd}tj|| ||z }|j ||g|}|j!|}|S)az Apply dynamic thresholding to the predicted sample. "Dynamic thresholding: At each sampling step we set s to a certain percentile absolute pixel value in xt0 (the prediction of x_0 at timestep t), and if s > 1, then we threshold xt0 to the range [-s, s] and then divide by s. Dynamic thresholding pushes saturated pixels (those near -1 and 1) inwards, thereby actively preventing pixels from saturation at each step. We find that dynamic thresholding results in significantly better photorealism as well as better image-text alignment, especially when using very large guidance weights." https://huggingface.co/papers/2205.11487 Args: sample (`torch.Tensor`): The predicted sample to be thresholded. Returns: `torch.Tensor`: The thresholded sample. r rX)r'max)r#shaper(r*float64floatreshaperqprodabsquantilerarDclamprE unsqueezery)rzr@r# batch_sizechannelsremaining_dims abs_sampless r_threshold_samplez(DEISMultistepScheduler._threshold_samplecs( 06 - H~  6 6\\^F Hrww~7N,NOZZ\ NN:t{{'M'MST U KK 1$++66  KKNVaR+a/ HF~F5! r rrctjtj|d}||ddtjfz }tj|dk\dj dj |jddz }|dz}||}||}||z ||z z } tj | dd} d| z |z| |zz} | j|j} | S)a Convert sigma values to corresponding timestep values through interpolation. Args: sigma (`np.ndarray`): The sigma value(s) to convert to timestep(s). log_sigmas (`np.ndarray`): The logarithm of the sigma schedule used for interpolation. Returns: `np.ndarray`: The interpolated timestep value(s) corresponding to the input sigma(s). g|=Nr)axisr)rr ) rqrlmaximumnewaxiscumsumargmaxcliprr) rzrr log_sigmadistslow_idxhigh_idxlowhighwrs rrz"DEISMultistepScheduler._sigma_to_tsFF2::eU34 Jq"**}55))UaZq188a8@EE*JZJZ[\J]`aJaEbQ;!(#9_t , GGAq! Ug H , IIekk "r cr|jjr d|z }|}||fSd|dzdzdzz }||z}||fS)a( Convert sigma values to alpha_t and sigma_t values. Args: sigma (`torch.Tensor`): The sigma value(s) to convert. Returns: `Tuple[torch.Tensor, torch.Tensor]`: A tuple containing (alpha_t, sigma_t) values. r rrV)rarL)rzrrjrks r_sigma_to_alpha_sigma_tz.DEISMultistepScheduler._sigma_to_alpha_sigma_tsY ;; & &%iGG E1HqLS01GgoGr rct|jdr|jj}nd}t|jdr|jj}nd}||n|dj }||n|dj }d}t j dd|}|d|z z}|d|z z}||||z zz|z} | S)a Construct the noise schedule as proposed in [Elucidating the Design Space of Diffusion-Based Generative Models](https://huggingface.co/papers/2206.00364). Args: in_sigmas (`torch.Tensor`): The input sigma values to be converted. num_inference_steps (`int`): The number of inference steps to generate the noise schedule for. Returns: `torch.Tensor`: The converted sigma values following the Karras noise schedule. sigma_minN sigma_maxr_rg@r )hasattrrarritemrqrO) rzrrprrrhoramp min_inv_rho max_inv_rhorns rrz)DEISMultistepScheduler._convert_to_karrass$ 4;; , --II 4;; , --II!*!6IIbMrCrrFrd) rzrr@argskwargsrrrjrkx0_preds rconvert_model_outputz+DEISMultistepScheduler.convert_model_outputCs,"$i!m47J1M >4y1}a !RSS   Z   DOO,77> ;; & &) 3, 66'AG [[ ( (H 4"G [[ ( (N :&<)??G [[ ( (,= =kk$//2Gw55G+DKK,G,G+HIWW  ;; # #,,W5G ;; % % /Ww..'9 9%&OP Pr ct|dkDr|dn|jdd}t|dkDr|dn|jdd}|t|dkDr|d}n td| tdd d | tdd d |j|j dz|j|j }}|j |\} }|j |\} }tj| tj|z } tj| tj|z } | | z } |jjd k(r)| | z |z|tj| d z z|zz }|Std)au One step for the first-order DEIS (equivalent to DDIM). Args: model_output (`torch.Tensor`): The direct output from the learned diffusion model. timestep (`int`): The current discrete timestep in the diffusion chain. prev_timestep (`int`): The previous discrete timestep in the diffusion chain. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. Returns: `torch.Tensor`: The sample tensor at the previous timestep. rrNr prev_timesteprrrtrrPassing `prev_timestep` is deprecated and has no effect as model output conversion is now handled via an internal counter `self.step_index`r2rWr) rrr$r rnr~rr(rlrarFrrd)rzrr@rrrrrksigma_srjalpha_srmlambda_shx_ts rdeis_first_order_updatez.DEISMultistepScheduler.deis_first_order_updates0"$i!m47J1M#&t9q=QfjjRV6W >4y1}a !RSS   Z   $ ^  KK!+ , KK ( 77@77@99W% '(::99W% '(:: x  ;; % % /W$.'UYYq\C=O2PT`1``C &&OP Pr model_output_listc.t|dkDr|dn|jdd}t|dkDr|dn|jdd}|t|dkDr|d}n td| tddd | tddd |j|j dz|j|j |j|j dz } }}|j |\} }|j |\} }|j | \} } |d |d }} || z || z | | z }}}|jjd k(rCd}||||||||z }||||||||z }| || z || zz||zzz}|Std)a One step for the second-order multistep DEIS. Args: model_output_list (`List[torch.Tensor]`): The direct outputs from learned diffusion model at current and latter timesteps. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. Returns: `torch.Tensor`: The sample tensor at the previous timestep. r timestep_listNr rrrrPassing `timestep_list` is deprecated and has no effect as model output conversion is now handled via an internal counter `self.step_index`rr_r2c|tj| tj|zdz ztj|tj|z z S)Nr rqrl)rbcs rind_fnzIDEISMultistepScheduler.multistep_deis_second_order_update..ind_fnsCRVVAYJ2Q67266!9rvvay;PQQr r rrr$r rnr~rrarFrd)rzrr@rrrrrksigma_s0sigma_s1rjalpha_s0alpha_s1m0m1rho_trho_s0rho_s1rcoef1coef2rs r"multistep_deis_second_order_updatez9DEISMultistepScheduler.multistep_deis_second_order_updates($'t9q=QfjjRV6W #&t9q=QfjjRV6W >4y1}a !RSS  $ ^   $ ^  KK!+ , KK ( KK!+ ,$  77@!99(C(!99(C("2&(9"(=B g  x  x v ;; % % / R5&&1F6664RRE5&&1F6664RREVh.;ebjHICJ%&OP Pr ct|dkDr|dn|jdd}t|dkDr|dn|jdd}|t|dkDr|d}n td| tddd | tddd |j|j dz|j|j |j|j dz |j|j dz f\}}} } |j |\} }|j |\} }|j | \} } |j | \}} |d |d |d }}}|| z || z | | z | |z f\}}}}|jjdk(rdd}||||||||||z }||||||||||z }||||||||||z }| || z ||zz||zz||zzz}|Std)a One step for the third-order multistep DEIS. Args: model_output_list (`List[torch.Tensor]`): The direct outputs from learned diffusion model at current and latter timesteps. sample (`torch.Tensor`): A current instance of a sample created by diffusion process. Returns: `torch.Tensor`: The sample tensor at the previous timestep. rrNr rrrrrrr_rr2c6|tj|tj|tj|z dzztj|tj|zz tj|ztj|dzzdtj|zz dzz}tj|tj|z tj|tj|z z}||z S)Nr rr)rrrd numerator denominators rrzHDEISMultistepScheduler.multistep_deis_third_order_update..ind_fnFsFF1IRVVAY!6!:;ffQi"&&)+,ffQi ffQi1n%"&&)m $    "vvay266!94RVVAY9NO  ;..r rr)rzrr@rrrrrkrrsigma_s2rjrralpha_s2rrm2rrrrho_s2rrrcoef3rs r!multistep_deis_third_order_updatez8DEISMultistepScheduler.multistep_deis_third_order_updatesf*$'t9q=QfjjRV6W #&t9q=QfjjRV6W >4y1}a !RSS  $ ^   $ ^  KK!+ , KK ( KK!+ , KK!+ , 1 -8X 77@!99(C(!99(C(!99(C(&r*,=b,ACTUWCXB g  x  x  x  ) %vvv ;; % % / /5&&&9F66SY[aA$($6$6q1u$=D  q ! >!-2 ;; # #q (D,A,AA,EIZ66|F6SK [[ % % *d.C.Ca.GK]AA$BTBT]cAdK@@ASAS\b@cK  4;;#;#; ;  ! !Q & ! A> !;77r c|S)a? Ensures interchangeability with schedulers that need to scale the denoising model input depending on the current timestep. Args: sample (`torch.Tensor`): The input sample. Returns: `torch.Tensor`: A scaled input sample. )rzr@rrs rscale_model_inputz(DEISMultistepScheduler.scale_model_inputs  r original_samplesnoisertc*|jj|j|j}|jjdk(rvt j |ra|jj|jt j}|j|jt j}n@|jj|j}|j|j}|j |Dcgc]}|j||}}nG|j|jg|jdz}n|jg|jdz}||j}t|jt|jkr=|jd}t|jt|jkr=|j!|\} } | |z| |zz} | Scc}w)a Add noise to the original samples according to the noise schedule at the specified timesteps. Args: original_samples (`torch.Tensor`): The original samples without noise. noise (`torch.Tensor`): The noise to add to the samples. timesteps (`torch.IntTensor`): The timesteps at which to add noise to the samples. Returns: `torch.Tensor`: The noisy samples. rmpsr"rr_)rnryrr#typer(is_floating_pointrtr*rr r~rflattenrrr) rzrrrtrnrr step_indicesrrjrk noisy_sampless r add_noisez DEISMultistepScheduler.add_noises,'7'>'>FVF\F\]  " " ' '5 0U5L5LY5W!%!2!23C3J3JRWR_R_!2!` ! %5%<%"22Wu_D _sHc.|jjSN)rar5r}s r__len__zDEISMultistepScheduler.__len__s{{...r )ig-C6?g{Gz?r9Nrr?Fgףp= ?rWr2r3TFFFFrWrOrFr4)r)NN)333333?r$r")T)/__name__ __module__ __qualname____doc__rname _compatiblesorderr intrrrrrqndarrayrboolr{propertyr~rrstrr(rrr rrrrrrrrrrrr r rrr IntTensorr r#).0es00rr1r1Osb;z%>>qAFF>L E$("QYBF[d",1"%*0)1"&,116*/*/&)GQ%*2?/], ],], ], MN ],  bjj$u+&= >? ],],!!WX],],%*], ], ],X&], ],$D>], !)!],""$#],$"$%],&UO'],(""CD)],*+],,#-],.!//],0 1],],~ HSM  !Xc]!!(3(t(,0" W, W,c5<<'(W, UO W,  W,t) ))X""" "J U\\ eELLRWR^R^D^>_ ,$ELL$s$W\WcWc$NTW\a\h\hFdg..8ll>8U\\)*>8 >8  >8 % & >8@  %,,  /,,/||/?? /  /b//K?sL>r1)g+?r)rtypingrrrrrnumpyrqr(configuration_utilsr r utilsr r scheduling_utilsrrr scipy.statsrr,rr r/r1rr rr:sx$ 88 A1XX 5=*4 *4*4"/2*4 \\ *4ZD/^[D/r