i ddlZddlmZmZmZmZmZddlZddl Z ddl m Z m Z ddl mZmZddlmZmZmZerddlZ dded ed ed d e j0fd Zde j0d e j0fdZGddee Zy)N)ListLiteralOptionalTupleUnion) ConfigMixinregister_to_config) deprecateis_scipy_available)KarrasDiffusionSchedulersSchedulerMixinSchedulerOutputnum_diffusion_timestepsmax_betaalpha_transform_type)cosineexpreturnc $|dk(rd}n|dk(rd}ntd|g}t|D]<}||z }|dz|z }|jtd||||z z |>t j |tj S)aB Create a beta schedule that discretizes the given alpha_t_bar function, which defines the cumulative product of (1-beta) over time from t = [0,1]. Contains a function alpha_bar that takes an argument t and transforms it to the cumulative product of (1-beta) up to that part of the diffusion process. Args: num_diffusion_timesteps (`int`): The number of betas to produce. max_beta (`float`, defaults to `0.999`): The maximum beta to use; use values lower than 1 to avoid numerical instability. alpha_transform_type (`"cosine"` or `"exp"`, defaults to `"cosine"`): The type of noise schedule for `alpha_bar`. Choose from `cosine` or `exp`. Returns: `torch.Tensor`: The betas used by the scheduler to step the model outputs. rcftj|dzdz tjzdz dzS)NgMb?gT㥛 ?r)mathcospits y/home/obispo/Crisostomo_bridge/mision_env/lib/python3.12/site-packages/diffusers/schedulers/scheduling_unipc_multistep.py alpha_bar_fnz)betas_for_alpha_bar..alpha_bar_fn<s-88QY%/$''9A=>!C Crc2tj|dzS)Ng()rrrs rrz)betas_for_alpha_bar..alpha_bar_fnAs88AI& &r z"Unsupported alpha_transform_type: r dtype) ValueErrorrangeappendmintorchtensorfloat32)rrrrbetasit1t2s rbetas_for_alpha_barr/"s0x' D  & '=>R=STUU E * +M ( (!e. . S\"- R0@@@(KLM <<U]] 33r r+c(d|z }tj|d}|j}|dj}|dj}||z}||||z z z}|dz}|dd|ddz }tj|dd|g}d|z }|S)a: Rescales betas to have zero terminal SNR Based on https://huggingface.co/papers/2305.08891 (Algorithm 1) Args: betas (`torch.Tensor`): The betas that the scheduler is being initialized with. Returns: `torch.Tensor`: Rescaled betas with zero terminal SNR. ?rdimrr N)r(cumprodsqrtclonecat)r+alphasalphas_cumprodalphas_bar_sqrtalphas_bar_sqrt_0alphas_bar_sqrt_T alphas_bars rrescale_zero_terminal_snrr?Ps5[F]]6q1N$))+O(*002'+113((O(,=@Q,QRRO!!#J ^j"o -F YY 1Q0 1F JE Lr c8zeZdZdZeDcgc]}|j c}}ZdZeddddddd d d d d dd gdd d d d d dddd d dfde de de de dde e ejee fde de ddede de ded e d!d"ed#ee d$e ed%e ed&e ed'e ed(e ed)e e d*e d+d,e d-e e d.d/ed0ed1e dd2df6d3Zed2e e fd4Zed2e e fd5Zd_d6e d2dfd7Z d`d8e d9e e eej4fd:e e d2dfd;Zdejd?ejd2ejfd@Zd>ej8d2eej8ej8ffdAZ dBej8d8e d2ej8fdCZ!dBej8d8e d2ej8fdDZ" dadBej8d8e dEe dFe d2ej8f dGZ#ddHdIej8dr4rY)midpointheunlogrhorD)rXcpu)+configr`r ImportErrorsumr_r^r$r(r)r*r+rEr/NotImplementedError __class__r?r9r5r:r6alpha_tsigma_tloglambda_tsigmasinit_noise_sigmar rWnum_inference_stepsnpcopy from_numpy timesteps model_outputs timestep_listlower_order_numsr\r] last_sample _step_index _begin_indexto)selfrHrIrJrKrNrOrPrTrUrVrWrXr[r\r]r^r_r`rarbrcrfrgrirjrkrs r__init__z UniPCMultistepScheduler.__init__s< ;; & &/A/CZ[ [  ++T[[-O-OQUQ\Q\QnQno pst tA   $m5==IDJ h & H>QY^YfYfgDJ o - C3H[chcpcpquvvDJ 1 1,-@ADJ%7OPTP^P^O_&`a a !24::>DJDJJ& #mmDKKQ? !'-D   #zz$"5"56 zz!d&9&9"9:  $,,/%))DLL2II D///43F3FF3N !$ n ,<<''E':)[M9QRVR`R`Qa*bcc$#' KK#6#:|jj.d k(rd} n"td|jj.tj0|| ggjtj2}n|jj4rtj&|}tj(|j}|j7|| }tj |Dcgc]}|j-||c}}|jj.d k(r|d} n>|jj.d k(rd} n"td|jj.tj0|| ggjtj2}ns|jj8rtj&|}tj(|j}|j;|| }tj |Dcgc]}|j-||c}}|jj.d k(r|d} n>|jj.d k(rd} n"td|jj.tj0|| ggjtj2}nJ|jj<rMtjdd|jjz |dz} d| z }tj(|jj |zd|jj dz |zzz ddj}||jjzj}|jj.d k(r|d} n>|jj.d k(rd} n"td|jj.tj0|| ggjtj2}ntj>|tjdtA||}|jj.d k(r&d|j"dz |j"dz d z} n>|jj.d k(rd} n"td|jj.tj0|| ggjtj2}tCjD||_#tCjD|jI|tBj|_%tA||_&dg|jjNz|_(d|_)d|_*|jVr'|jVjY|jL|d|_-d|_.|jFjId|_#ycc}wcc}wcc}w)a6 Sets the discrete timesteps used for the diffusion chain (to be run before inference). Args: num_inference_steps (`int`): The number of diffusion steps used when generating samples with a pre-trained model. device (`str` or `torch.device`, *optional*): The device to which the timesteps should be moved to. If `None`, the timesteps are not moved. mu (`float`, *optional*): Optional mu parameter for dynamic shifting when using exponential time shift type. NrGrErr r4rdrezY is not supported. Please make sure to choose one of 'linspace', 'leading' or 'trailing'.rm) in_sigmasr}rhrFzC`final_sigmas_type` must be one of 'zero', or 'sigma_min', but got r1rr#rrq)/rrrjrkr~rrbrcrErHroundrastypeint64arangerfr$arrayr:r^ryflip_convert_to_karras _sigma_to_trg concatenater*r__convert_to_exponentialr`_convert_to_betarainterplenr(rr{rrr}rOrrrr] set_timestepsrr) rr}rrr step_ratior{ log_sigmassigma sigma_lastr9s rrz%UniPCMultistepScheduler.set_timesteps2s >;;33 8S8SWd8d dd%'VVBZDKK " ;; ' ': 5 At{{>>BDWZ[D[\2"$!  [[ ) )Y 688=PST=TUJ1&9A&=>KRRTUYWYUYZ[^\^_ddfmmnpnvnvwI 11 1I [[ ) )Z 788;NNJ $++"A"A1zkRXXZ__ahhikiqiqrI NI;;//01JK A 3 33t7J7JJsRS ;; ( (JWWV_))+F,,vSf,gFSY!Z%$"2"25*"E!Z[aacI{{,, ;#BZ ..&8  YZ^ZeZeZwZwYxy^^Vj\$:;BB2::NF [[ / /JWWV_))+F11FXk1lFSY!Z%$"2"25*"E!Z[I{{,, ;#BZ ..&8  YZ^ZeZeZwZwYxy^^Vj\$:;BB2::NF [[ ( (JWWV_))+F**VQd*eFSY!Z%$"2"25*"E!Z[I{{,, ;#BZ ..&8  YZ^ZeZeZwZwYxy^^Vj\$:;BB2::NF [[ ( ([[A (G(G$GI\_`I`aF6\FWWT[[33f<T[[E[E[^_E_ciDi@ijklomopuuwF$++"A"AAGGII{{,, ;#BZ ..&8  YZ^ZeZeZwZwYxy^^Vj\$:;BB2::NFYYy"))As6{*CVLF{{,, ; 4#6#6q#99T=P=PQR=SSX[[ ..&8  YZ^ZeZeZwZwYxy^^Vj\$:;BB2::NF&&v. )))477vU[[7Y#&y>   KK $ $%!" == MM ' '(@(@ ' P  kknnU+ a"["["[s9e 0eerQcb|j}|j^}}}|tjtjfvr|j }|j ||tj|z}|j}tj||jjd}tj|d|jj}|jd}tj|| ||z }|j ||g|}|j!|}|S)az Apply dynamic thresholding to the predicted sample. "Dynamic thresholding: At each sampling step we set s to a certain percentile absolute pixel value in xt0 (the prediction of x_0 at timestep t), and if s > 1, then we threshold xt0 to the range [-s, s] and then divide by s. Dynamic thresholding pushes saturated pixels (those near -1 and 1) inwards, thereby actively preventing pixels from saturation at each step. We find that dynamic thresholding results in significantly better photorealism as well as better image-text alignment, especially when using very large guidance weights." https://huggingface.co/papers/2205.11487 Args: sample (`torch.Tensor`): The predicted sample to be thresholded. Returns: `torch.Tensor`: The thresholded sample. r r2)r'max)r#shaper(r*float64floatreshaper~prodabsquantilerrrUclamprV unsqueezer)rrQr# batch_sizechannelsremaining_dims abs_sampless r_threshold_samplez)UniPCMultistepScheduler._threshold_samples( 06 - H~  6 6\\^F Hrww~7N,NOZZ\ NN:t{{'M'MST U KK 1$++66  KKNVaR+a/ HF~F5! r rrctjtj|d}||ddtjfz }tj|dk\dj dj |jddz }|dz}||}||}||z ||z z } tj | dd} d| z |z| |zz} | j|j} | S)a Convert sigma values to corresponding timestep values through interpolation. Args: sigma (`np.ndarray`): The sigma value(s) to convert to timestep(s). log_sigmas (`np.ndarray`): The logarithm of the sigma schedule used for interpolation. Returns: `np.ndarray`: The interpolated timestep value(s) corresponding to the input sigma(s). g|=Nr)axisr)rr ) r~rymaximumnewaxiscumsumargmaxcliprr) rrr log_sigmadistslow_idxhigh_idxlowhighwrs rrz#UniPCMultistepScheduler._sigma_to_tsFF2::eU34 Jq"**}55))UaZq188a8@EE*JZJZ[\J]`aJaEbQ;!(#9_t , GGAq! Ug H , IIekk "r cr|jjr d|z }|}||fSd|dzdzdzz }||z}||fS)a( Convert sigma values to alpha_t and sigma_t values. Args: sigma (`torch.Tensor`): The sigma value(s) to convert. Returns: `Tuple[torch.Tensor, torch.Tensor]`: A tuple containing (alpha_t, sigma_t) values. r rrm)rrra)rrrwrxs r_sigma_to_alpha_sigma_tz/UniPCMultistepScheduler._sigma_to_alpha_sigma_tsY ;; & &%iGG E1HqLS01GgoGr rct|jdr|jj}nd}t|jdr|jj}nd}||n|dj }||n|dj }d}t j dd|}|d|z z}|d|z z}||||z zz|z} | S)a Construct the noise schedule as proposed in [Elucidating the Design Space of Diffusion-Based Generative Models](https://huggingface.co/papers/2206.00364). Args: in_sigmas (`torch.Tensor`): The input sigma values to be converted. num_inference_steps (`int`): The number of inference steps to generate the noise schedule for. Returns: `torch.Tensor`: The converted sigma values following the Karras noise schedule. rhN sigma_maxr4rg@r )hasattrrrrhritemr~rE) rrr}rhrrhoramp min_inv_rho max_inv_rhor{s rrz*UniPCMultistepScheduler._convert_to_karrass$ 4;; , --II 4;; , --II!*!6IIbM4y1}a !RSS   Z   DOO,77> ??{{**i7!Gl$::gE,,8&,,>!F*W|-CC,,0AA++doo6 7\#99 / 0K0K/LM\\ {{''009N{{**i7##,,8!Gl$::gE,,>!L07V3CC / 0K0K/LMGGr )rQorderrcz t|dkDr|dn|jdd}|t|dkDr|d}n td|t|dkDr|d}n td| tddd |j}|j d }|d } |} |j r)|j j||| j} | S|j|jdz|j|j} } |j| \}} |j| \}} tj|tj| z }tj|tj| z }||z }|j}g}g}td|D]}|j|z }||dz }|j|j|\}}tj|tj|z }||z |z }|j!||j!|| z |z |j!d tj"|| }g}g}|j$r| n|}tj&|} | |z dz }!d}"|j(j*d k(r|}#n9|j(j*dk(rtj&|}#n t-td|dzD]T}|j!tj.||dz |j!|!|"z|#z |"|dzz}"|!|z d|"z z }!Vtj0|}tj"|| }t|dkDrtj0|d}|dk(r$tj"dg| j2|}$nWtj4j7|dd dd f|dd j9|j9| j2}$nd}|j$r9| | z | z|| z| zz }%|tj:d$|}&nd}&|%||#z|&zz } n8||z | z| | z| zz }%|tj:d$|}&nd}&|%| |#z|&zz } | j9| j2} | S)a One step for the UniP (B(h) version). Alternatively, `self.solver_p` is used if is specified. Args: model_output (`torch.Tensor`): The direct output from the learned diffusion model at the current timestep. prev_timestep (`int`): The previous discrete timestep in the diffusion chain. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. order (`int`): The order of UniP at this timestep (corresponds to the *p* in UniPC-p). Returns: `torch.Tensor`: The sample tensor at the previous timestep. r prev_timestepNr rr.missing `order` as a required keyword argumentrzPassing `prev_timestep` is deprecated and has no effect as model output conversion is now handled via an internal counter `self.step_index`r4r1rrZrDr2rmr#rk,bkc...->bc...)rrr$r rrr]step prev_sampler{rrr(ryrr%r&r)rWexpm1rrrXrupowstackr#linalgsolvereinsum)'rrrQrrrrmodel_output_lists0m0xx_trxsigma_s0rwalpha_s0rz lambda_s0hrrksD1sr,simialpha_sisigma_si lambda_sirkRbhhh_phi_1h_phi_k factorial_iB_hrhos_px_t_pred_ress' rmultistep_uni_p_bh_updatez1UniPCMultistepScheduler.multistep_uni_p_bh_updates~2$'t9q=QfjjRV6W >4y1}a !RSS =4y1}Q !QRR  $ ^  !..    # r "  ==--$$\2q9EECJ KK!(;Z77@!99(C(99W% '(::IIh'%))H*== y q% 'A1$B"QU8,B!%!=!=dkk"o!N Hh (+eii.AAIi'1,B JJrN JJR2~ & ' 3ll3v.  ??aR++b/B," ;; " "e +C [[ $ $ -++b/C%' 'q%!)$ 5A HHUYYsAE* + HHW{*S0 1 1q5 KlQ_4G  5 KKN LL6 * s8a<++cq)Czse1776J++Acrc3B3hK3B@CCFKNNqwwWC ??X%)Gg,=,BBD <<(963G311CX%)Gg,=,BBD <<(963G311CffQWWo r )r this_samplerthis_model_outputrrcn t|dkDr|dn|jdd}|t|dkDr|d}n td|t|dkDr|d}n td|t|dkDr|d}n td | tdd d |j}|d } |} |} |} |j |j |j |j dz }} |j| \}} |j|\}}tj|tj| z }tj|tj|z }||z }|j}g}g}td|D]}|j |dzz }||dz }|j|j |\}}tj|tj|z }||z |z }|j||j|| z |z |jd tj||}g}g}|jr| n|} tj| }!|!| z dz }"d}#|j j"dk(r| }$n9|j j"dk(rtj| }$n t%td|dzD]T}|jtj&||dz |j|"|#z|$z |#|dzz}#|"| z d|#z z }"Vtj(|}tj||}t|dkDrtj(|d}nd}|dk(r$tjdg| j*|}%nHtj,j/||j1|j1| j*}%|jrJ| |z | z||!z| zz }&|tj2d|%dd |}'nd}'| | z }(|&||$z|'|%d |(zzzz } nI||z | z| |!z| zz }&|tj2d|%dd |}'nd}'| | z }(|&| |$z|'|%d |(zzzz } | j1| j*} | S)a One step for the UniC (B(h) version). Args: this_model_output (`torch.Tensor`): The model outputs at `x_t`. this_timestep (`int`): The current timestep `t`. last_sample (`torch.Tensor`): The generated sample before the last predictor `x_{t-1}`. this_sample (`torch.Tensor`): The generated sample after the last predictor `x_{t}`. order (`int`): The `p` of UniC-p at this step. The effective order of accuracy should be `order + 1`. Returns: `torch.Tensor`: The corrected sample tensor at the current timestep. r this_timestepNr z4missing `last_sample` as a required keyword argumentrz4missing `this_sample` as a required keyword argumentrrzPassing `this_timestep` is deprecated and has no effect as model output conversion is now handled via an internal counter `self.step_index`r4r1rrZrDr2rmrr)rrr$r rr{rrr(ryrr%r&r)rWrrrrXrurrr#rrrr))rrrrrrrrrrrrmodel_trxrrwrrzrrrrrr,rrrrrrrrrrr r r rhos_cr corr_resD1_ts) rmultistep_uni_c_bh_updatez1UniPCMultistepScheduler.multistep_uni_c_bh_update^s8$'t9q=QfjjRV6W  4y1}"1g  !WXX  4y1}"1g  !WXX =4y1}Q !QRR  $ ^  !.. r " # KK8$++dooXYFY:Z77@!99(C(99W% '(::IIh'%))H*== y ##q% 'AAE*B"QU8,B!%!=!=dkk"o!N Hh (+eii.AAIi'1,B JJrN JJR2~ & ' 3ll3v.  ??aR++b/B," ;; " "e +C [[ $ $ -++b/C%' 'q%!)$ 5A HHUYYsAE* + HHW{*S0 1 1q5 KlQ_4G  5 KKN LL6 * s8a<++cq)CC A:\\3%qwwvFF\\''1-008;;AGGDF ??X%)Gg,=,BBD <<(96#2;LRA$($6$6q1u$=D  q !$($6$6q1u$=D  q ! >"62!)2 ;; ( (T[[55s4>>7JT__7\]J11Jj$*?*?!*CD"""!44%//5  4;;#;#; ;  ! !Q & ! A> !;77r c|S)a? Ensures interchangeability with schedulers that need to scale the denoising model input depending on the current timestep. Args: sample (`torch.Tensor`): The input sample. Returns: `torch.Tensor`: A scaled input sample. )rrQrrs rscale_model_inputz)UniPCMultistepScheduler.scale_model_inputls  r original_samplesnoiserc*|jj|j|j}|jjdk(rvt j |ra|jj|jt j}|j|jt j}n@|jj|j}|j|j}|j |Dcgc]}|j||}}nG|j|jg|jdz}n|jg|jdz}||j}t|jt|jkr=|jd}t|jt|jkr=|j!|\} } | |z| |zz} | Scc}w)a Add noise to the original samples according to the noise schedule at the specified timesteps. Args: original_samples (`torch.Tensor`): The original samples without noise. noise (`torch.Tensor`): The noise to add to the samples. timesteps (`torch.IntTensor`): The timesteps at which to add noise to the samples. Returns: `torch.Tensor`: The noisy samples. rmpsr"rr4)r{rrr#typer(is_floating_pointrr*rrrrflattenrrr) rr+r,rr{rr step_indicesrrwrx noisy_sampless r add_noisez!UniPCMultistepScheduler.add_noise|s,'7'>'>FVF\F\]  " " ' '5 0U5L5LY5W!%!2!23C3J3JRWR_R_!2!` ! %5%<%"22Wu_D _sHc.|jjSN)rrrHrs r__len__zUniPCMultistepScheduler.__len__s{{...r )r)NN)333333?r8r6)T)/__name__ __module__ __qualname____doc__rname _compatiblesrr intrrrrr~ndarrayrboolrrpropertyrrrstrr(rrr!rrrrrrrrrrrr"rrr* IntTensorr4r7).0es00rrArAtsEN%>>qAFF>L E$("QYBF[d",1"%-2"&')-1,116*/*/&)GQDJ',%*2?7W, W,W, W, MN W,  bjj$u+&= >? W,W,!!WXW,W,%*W, W,W,\*W, W, 9W, >*!W,"$D>#W,$!)%W,&"$'W,("$)W,*UO+W,,""CD-W,./W,0$G,?$@A1W,2!%3W,4#5W,6!/7W,8 9W,W,r HSM  !Xc]!!(3(t(rv,#&,08sELL?P9Q0R,_ghm_n, ,D) ))X""" "J U\\ eELLRWR^R^D^>_ ,$ELL$s$W\WcWc$NTW\a\h\hFdg..rMs$ 88 A1XX 5=*4 *4*4"/2*4 \\ *4\!U\\!ell!Hz/nkz/r