i ddlZddlmZmZmZmZmZmZddlZ ddl Z ddl m Z m Z ddlmZmZddlmZddlmZmZmZerddlZ dd ed ed ed d e j6fdZGddee Zy)N)CallableListLiteralOptionalTupleUnion) ConfigMixinregister_to_config) deprecateis_scipy_available) randn_tensor)KarrasDiffusionSchedulersSchedulerMixinSchedulerOutputnum_diffusion_timestepsmax_betaalpha_transform_type)cosineexpreturnc $|dk(rd}n|dk(rd}ntd|g}t|D]<}||z }|dz|z }|jtd||||z z |>t j |tj S)aB Create a beta schedule that discretizes the given alpha_t_bar function, which defines the cumulative product of (1-beta) over time from t = [0,1]. Contains a function alpha_bar that takes an argument t and transforms it to the cumulative product of (1-beta) up to that part of the diffusion process. Args: num_diffusion_timesteps (`int`): The number of betas to produce. max_beta (`float`, defaults to `0.999`): The maximum beta to use; use values lower than 1 to avoid numerical instability. alpha_transform_type (`"cosine"` or `"exp"`, defaults to `"cosine"`): The type of noise schedule for `alpha_bar`. Choose from `cosine` or `exp`. Returns: `torch.Tensor`: The betas used by the scheduler to step the model outputs. rcftj|dzdz tjzdz dzS)NgMb?gT㥛 ?r )mathcospits r/home/obispo/Crisostomo_bridge/mision_env/lib/python3.12/site-packages/diffusers/schedulers/scheduling_sasolver.py alpha_bar_fnz)betas_for_alpha_bar..alpha_bar_fn=s-88QY%/$''9A=>!C Crc2tj|dzS)Ng()rrrs r r!z)betas_for_alpha_bar..alpha_bar_fnBs88AI& &r"z"Unsupported alpha_transform_type: rdtype) ValueErrorrangeappendmintorchtensorfloat32)rrrr!betasit1t2s r betas_for_alpha_barr1#s0x' D  & '=>R=STUU E * +M ( (!e. . S\"- R0@@@(KLM <<U]] 33r"c0eZdZdZeDcgc]}|j c}}ZdZedddddddd dd d d d dd d d d d e d dddfde de de de de e ejee fde de de de edede de de ded e ed!e ed"e ed#e ed$e e d%e d&e e d'e d(e f.d)Zed*Zed+ZdWd,e fd-ZdXd.e d/e e ej2ffd0Zd1ej6d2ej6fd3Zd4Zd5Zd6ej6d2ej6fd7Zd6ej6d.e d2ej6fd8Z dYd6ej6d.e d9e d:e d2ej6f d;Z!dd<d=ej6d1ej6d2ej6fd>Z"d?Z#d@Z$dAZ%dBZ&d=ej6d1ej6dCej6dDe dEej6d2ej6f dFZ'dGej6dHej6dIej6dJej6dDe dEej6d2ej6fdKZ( dZdLe e ej6fdMe ej6d2e fdNZ)dOZ* d[d=ej6dLe d1ej6dPed2e e+e,ff dQZ-d1ej6d2ej6fdRZ.dSej6dCej6dTej^d2ej6fdUZ0dVZ1ycc}}w)\SASolverScheduleru `SASolverScheduler` is a fast dedicated high-order solver for diffusion SDEs. This model inherits from [`SchedulerMixin`] and [`ConfigMixin`]. Check the superclass documentation for the generic methods the library implements for all schedulers such as loading and saving. Args: num_train_timesteps (`int`, defaults to 1000): The number of diffusion steps to train the model. beta_start (`float`, defaults to 0.0001): The starting `beta` value of inference. beta_end (`float`, defaults to 0.02): The final `beta` value. beta_schedule (`str`, defaults to `"linear"`): The beta schedule, a mapping from a beta range to a sequence of betas for stepping the model. Choose from `linear`, `scaled_linear`, or `squaredcos_cap_v2`. trained_betas (`np.ndarray`, *optional*): Pass an array of betas directly to the constructor to bypass `beta_start` and `beta_end`. predictor_order (`int`, defaults to 2): The predictor order which can be `1` or `2` or `3` or '4'. It is recommended to use `predictor_order=2` for guided sampling, and `predictor_order=3` for unconditional sampling. corrector_order (`int`, defaults to 2): The corrector order which can be `1` or `2` or `3` or '4'. It is recommended to use `corrector_order=2` for guided sampling, and `corrector_order=3` for unconditional sampling. prediction_type (`str`, defaults to `epsilon`, *optional*): Prediction type of the scheduler function; can be `epsilon` (predicts the noise of the diffusion process), `sample` (directly predicts the noisy sample`) or `v_prediction` (see section 2.4 of [Imagen Video](https://huggingface.co/papers/2210.02303) paper). tau_func (`Callable`, *optional*): Stochasticity during the sampling. Default in init is `lambda t: 1 if t >= 200 and t <= 800 else 0`. SA-Solver will sample from vanilla diffusion ODE if tau_func is set to `lambda t: 0`. SA-Solver will sample from vanilla diffusion SDE if tau_func is set to `lambda t: 1`. For more details, please check https://huggingface.co/papers/2309.05019 thresholding (`bool`, defaults to `False`): Whether to use the "dynamic thresholding" method. This is unsuitable for latent-space diffusion models such as Stable Diffusion. dynamic_thresholding_ratio (`float`, defaults to 0.995): The ratio for the dynamic thresholding method. Valid only when `thresholding=True`. sample_max_value (`float`, defaults to 1.0): The threshold value for dynamic thresholding. Valid only when `thresholding=True` and `algorithm_type="dpmsolver++"`. algorithm_type (`str`, defaults to `data_prediction`): Algorithm type for the solver; can be `data_prediction` or `noise_prediction`. It is recommended to use `data_prediction` with `solver_order=2` for guided sampling like in Stable Diffusion. lower_order_final (`bool`, defaults to `True`): Whether to use lower-order solvers in the final steps. Default = True. use_karras_sigmas (`bool`, *optional*, defaults to `False`): Whether to use Karras sigmas for step sizes in the noise schedule during the sampling process. If `True`, the sigmas are determined according to a sequence of noise levels {σi}. use_exponential_sigmas (`bool`, *optional*, defaults to `False`): Whether to use exponential sigmas for step sizes in the noise schedule during the sampling process. use_beta_sigmas (`bool`, *optional*, defaults to `False`): Whether to use beta sigmas for step sizes in the noise schedule during the sampling process. Refer to [Beta Sampling is All You Need](https://huggingface.co/papers/2407.12173) for more information. lambda_min_clipped (`float`, defaults to `-inf`): Clipping threshold for the minimum value of `lambda(t)` for numerical stability. This is critical for the cosine (`squaredcos_cap_v2`) noise schedule. variance_type (`str`, *optional*): Set to "learned" or "learned_range" for diffusion models that predict variance. If set, the model's output contains the predicted Gaussian variance. timestep_spacing (`str`, defaults to `"linspace"`): The way the timesteps should be scaled. Refer to Table 2 of the [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://huggingface.co/papers/2305.08891) for more information. steps_offset (`int`, defaults to 0): An offset added to the inference steps, as required by some model families. rig-C6?g{Gz?linearNr epsilonFgףp= ??data_predictionTinflinspacernum_train_timesteps beta_startbeta_end beta_schedule trained_betaspredictor_ordercorrector_orderprediction_typetau_func thresholdingdynamic_thresholding_ratiosample_max_valuealgorithm_typelower_order_finaluse_karras_sigmasuse_exponential_sigmasuse_beta_sigmasuse_flow_sigmas flow_shiftlambda_min_clipped variance_typetimestep_spacing steps_offsetc|jjrts tdt |jj|jj |jj gdkDr td|+tj|tj|_ n|dk(r-tj|||tj|_ nk|dk(r6tj|dz|dz|tjdz|_ n0|d k(rt||_ nt|d |jd |jz |_tj"|j d |_tj&|j$|_tj&d|j$z |_tj,|j(tj,|j*z |_d|j$z |j$z dz|_d |_| dvrt| d |jd|_t7jd |dz |t6jdddj9}tj:||_dgt?||dz z|_ dgt?||dz z|_!| d|_"n| |_"| dk(|_#d |_$d|_%d|_&d|_'|j0jQd|_y)Nz:Make sure to install scipy if you want to use beta sigmas.rznOnly one of `config.use_beta_sigmas`, `config.use_exponential_sigmas`, `config.use_karras_sigmas` can be used.r$r4 scaled_linear?r squaredcos_cap_v2z is not implemented for r6rdim)r7noise_predictionc|dk\r|dkrdSdS)Ni rrrs r z,SASolverScheduler.__init__..s18Saar"r7cpu))configrJr ImportErrorsumrIrHr&r*r+r,r-r9r1NotImplementedError __class__alphascumprodalphas_cumprodsqrtalpha_tsigma_tloglambda_tsigmasinit_noise_sigmanum_inference_stepsnpcopy from_numpy timestepsmax timestep_list model_outputsrB predict_x0lower_order_nums last_sample _step_index _begin_indexto)selfr:r;r<r=r>r?r@rArBrCrDrErFrGrHrIrJrKrLrMrNrOrPrqs r __init__zSASolverScheduler.__init__s6 ;; & &/A/CZ[ [  ++T[[-O-OQUQ\Q\QnQno pst tA   $m5==IDJ h & H>QY^YfYfgDJ o -OcM'--    J1 1,-@ADJ%7OPTP^P^O_&`a aDJJ& #mmDKKQ?zz$"5"56 zz!d&9&9"9:  $,,/%))DLL2II D///43F3FF3N !$ !H H%(88PQUQ_Q_P`&ab b$( KK#6#:rtjdd|jj z |dz} d | z }tj|jj@|zd|jj@dz |zzz ddj}||jj zj}tj2||ddgjtj4}ntjB|tj dtE||}d|j(dz |j(dz d z} tj2|| ggjtj4}tjF||_$tjF|jK|tj |_&tE||_'dgtQ|jjR|jjTdz z|_+d|_,d|_-d|_.d|_/|jHjKd |_$ycc} wcc} wcc} w)a Sets the discrete timesteps used for the diffusion chain (to be run before inference). Args: num_inference_steps (`int`): The number of diffusion steps used when generating samples with a pre-trained model. device (`str` or `torch.device`, *optional*): The device to which the timesteps should be moved to. If `None`, the timesteps are not moved. rr9rNrXleadingtrailingzY is not supported. Please make sure to choose one of 'linspace', 'leading' or 'trailing'.rS) in_sigmasrmr6)rr%r])0r* searchsortedfliprjr^rMr:numpyitemrOrnr9roundroastypeint64arangerPr&arrayrerirH_convert_to_karras _sigma_to_t concatenater,rI_convert_to_exponentialrJ_convert_to_betarKrLinterplenrprkrzrqrmrrr?r@rtrvrwrxry) r{rmr clipped_idx last_timesteprq step_ratiork log_sigmassigmarc sigma_lasts r set_timestepszSASolverScheduler.set_timestepss((DMMA3)GIgIgh ++99KGNNPVVX  ;; ' ': 5 A}q02E2IJPPRSWUWSWXY\Z\]bbdkklnltltu [[ ) )Y 6&+>+BCJ1&9A&=>KRRTUYWYUYZ[^\^_ddfmmnpnvnvwI 11 1I [[ ) )Z 788;NNJ -ZK@FFHMMOVVWYW_W_`I NI;;//01JK A 3 33t7J7JJsRSVVF^ ;; ( (WWV_))+F,,vSf,gFSY!Z%$"2"25*"E!Z[aacI^^VVBC[$9:AA"**MF [[ / /WWV_))+F11FXk1lFSY!Z%$"2"25*"E!Z[I^^VVBC[$9:AA"**MF [[ ( (WWV_))+F**VQd*eFSY!Z%$"2"25*"E!Z[I^^VVBC[$9:AA"**MF [[ ( ([[A (G(G$GI\_`I`aF6\FWWT[[33f<T[[E[E[^_E_ciDi@ijklomopuuwF$++"A"AAGGII^^VVBC[$9:AA"**MFYYy"))As6{*CVLFt221559L9LQ9OOTWWJ^^Vj\$:;BB2::NF&&v. )))477vU[[7Y#&y>   ++T[[-H-H1-L MN!"  kknnU+ I"[ "[ "[s#\0)\5!\:samplercb|j}|j^}}}|tjtjfvr|j }|j ||tj|z}|j}tj||jjd}tj|d|jj}|jd}tj|| ||z }|j ||g|}|j!|}|S)az Apply dynamic thresholding to the predicted sample. "Dynamic thresholding: At each sampling step we set s to a certain percentile absolute pixel value in xt0 (the prediction of x_0 at timestep t), and if s > 1, then we threshold xt0 to the range [-s, s] and then divide by s. Dynamic thresholding pushes saturated pixels (those near -1 and 1) inwards, thereby actively preventing pixels from saturation at each step. We find that dynamic thresholding results in significantly better photorealism as well as better image-text alignment, especially when using very large guidance weights." https://huggingface.co/papers/2205.11487 Args: sample (`torch.Tensor`): The predicted sample to be thresholded. Returns: `torch.Tensor`: The thresholded sample. rrU)r)rr)r%shaper*r,float64floatreshapernprodabsquantiler^rDclamprE unsqueezerz)r{rr% batch_sizechannelsremaining_dims abs_sampless r _threshold_samplez#SASolverScheduler._threshold_sampleXs( 06 - H~  6 6\\^F Hrww~7N,NOZZ\ NN:t{{'M'MST U KK 1$++66  KKNVaR+a/ HF~F5! r"ctjtj|d}||ddtjfz }tj|dk\dj dj |jddz }|dz}||}||}||z ||z z } tj | dd} d| z |z| |zz} | j|j} | S)a Convert sigma values to corresponding timestep values through interpolation. Args: sigma (`np.ndarray`): The sigma value(s) to convert to timestep(s). log_sigmas (`np.ndarray`): The logarithm of the sigma schedule used for interpolation. Returns: `np.ndarray`: The interpolated timestep value(s) corresponding to the input sigma(s). g|=Nr)axisr )rrr) rnrimaximumnewaxiscumsumargmaxcliprr) r{rr log_sigmadistslow_idxhigh_idxlowhighwrs r rzSASolverScheduler._sigma_to_tsFF2::eU34 Jq"**}55))UaZq188a8@EE*JZJZ[\J]`aJaEbQ;!(#9_t , GGAq! Ug H , IIekk "r"cr|jjr d|z }|}||fSd|dzdzdzz }||z}||fS)a( Convert sigma values to alpha_t and sigma_t values. Args: sigma (`torch.Tensor`): The sigma value(s) to convert. Returns: `Tuple[torch.Tensor, torch.Tensor]`: A tuple containing (alpha_t, sigma_t) values. rr rS)r^rK)r{rrgrhs r _sigma_to_alpha_sigma_tz)SASolverScheduler._sigma_to_alpha_sigma_tsY ;; & &%iGG E1HqLS01GgoGr"rct|jdr|jj}nd}t|jdr|jj}nd}||n|dj }||n|dj }d}t j dd|}|d|z z}|d|z z}||||z zz|z} | S)a Construct the noise schedule as proposed in [Elucidating the Design Space of Diffusion-Based Generative Models](https://huggingface.co/papers/2206.00364). Args: in_sigmas (`torch.Tensor`): The input sigma values to be converted. num_inference_steps (`int`): The number of inference steps to generate the noise schedule for. Returns: `torch.Tensor`: The converted sigma values following the Karras noise schedule. sigma_minN sigma_maxrXrg@r)hasattrr^rrrrnr9) r{rrmrrrhoramp min_inv_rho max_inv_rhorks r rz$SASolverScheduler._convert_to_karrass$ 4;; , --II 4;; , --II!*!6IIbM [!TIP] > The algorithm and model type are decoupled. You can use either data_prediction or noise_prediction for both > noise prediction and data prediction models. Args: model_output (`torch.Tensor`): The direct output from the learned diffusion model. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. Returns: `torch.Tensor`: The converted model output. rrNr/missing `sample` as a required keyword argumentrq1.0.0zPassing `timesteps` is deprecated and has no effect as model output conversion is now handled via an internal counter `self.step_index`)r7r5)learned learned_ranger v_predictionflow_predictionzprediction_type given as zd must be one of `epsilon`, `sample`, `v_prediction`, or `flow_prediction` for the SASolverScheduler.)rWzQ must be one of `epsilon`, `sample`, or `v_prediction` for the SASolverScheduler.)rpopr&r rkrrr^rFrArNrCrrgrh) r{rrargskwargsrrrgrhx0_predr5s r convert_model_outputz&SASolverScheduler.convert_model_output8s2"$i!m47J1M >4y1}a !RSS   Z   DOO,77> ;; % %)< <{{**i7;;,,0LL#/2A2#6L!Gl$::gE,,8&,,>!F*W|-CC,,0AA++doo6 7\#99 / 0K0K/LMVV {{''009N[[ ' '+? ?{{**i7;;,,0LL*1bqb51G*G,,8!Gl$::gE,,>!L07V3CC / 0K0K/LMAA {{''#'<<#94<<;Q!Gg$55@009!Gg$55@N/@r"ct|dvsJd|dk(r2tj| tj||z dz zS|dk(r;tj| |dztj||z z|dzz zS|dk(rMtj| |dzd|zzdztj||z z|dzd|zzdzz zS|dk(r_tj| |dzd|dzzzd|zzdztj||z z|dzd|dzzzd|zzdzz zSy) zd Calculate the integral of exp(-x) * x^order dx from interval_start to interval_end rrr r)order is only supported for 0, 1, 2 and 3rrr rNr*r)r{orderinterval_start interval_ends r %get_coefficients_exponential_negativez7SASolverScheduler.get_coefficients_exponential_negatives $Q&QQ$ A:99l]+uyy9V/WZ[/[\ \ aZ99l]+!#uyy1N'OOS_bcScd aZ99l]+"Q%77!;uyyXfIf?gg?Q%559; aZ99l]+"Q):%::Q=OORSS))L>9:;?Qq%881|;KKaOQ r"c|dvsJdd|dzz|z}d|dzz|z}|dk(r;tj|dtj||z z zd|dzzz S|dk(rGtj||dz |dz tj||z zz zd|dzzdzz S|dk(rYtj||dzd|zz dz|dzd|zz dztj||z zz zd|dzzdzz S|dk(rktj||dzd|dzzz d|zzdz |dzd|dzzz d|zzdz tj||z zz zd|dzzdzz Sy ) zl Calculate the integral of exp(x(1+tau^2)) * x^order dx from interval_start to interval_end rrrr rrrNr)r{rrrtauinterval_end_covinterval_start_covs r %get_coefficients_exponential_positivez7SASolverScheduler.get_coefficients_exponential_positives& $Q&QQ$QJ,6#q&jN: A: *+q599?ORd?d=e3f/fgklortuoukuv aZ *+%))A-=MPb=b;c1dde QJ1$ & aZ *+%q(1/?+??!C)1,q3E/EEIii"25G"G HIJJ QJ1$ & aZ *+%q(1/?/B+BBQIYEYY\]])1,q3Eq3H/HH1OaKaadeeii"25G"G HIJJ QJ1$ & r"c |dvsJ|t|dz k(sJ|dk(rdggS|dk(r@d|d|dz z |d |d|dz z gd|d|dz z |d |d|dz z ggS|dk(r|d|dz |d|dz z}|d|dz |d|dz z}|d|dz |d|dz z}d|z |d |dz |z |d|dz|z gd|z |d |dz |z |d|dz|z gd|z |d |dz |z |d|dz|z ggS|dk(r|d|dz |d|dz z|d|dz z}|d|dz |d|dz z|d|dz z}|d|dz |d|dz z|d|dz z}|d|dz |d|dz z|d|dz z}d|z |d |dz |dz |z |d|dz|d|dzz|d|dzz|z |d |dz|dz|z gd|z |d |dz |dz |z |d|dz|d|dzz|d|dzz|z |d |dz|dz|z gd|z |d |dz |dz |z |d|dz|d|dzz|d|dzz|z |d |dz|dz|z gd|z |d |dz |dz |z |d|dz|d|dzz|d|dzz|z |d |dz|dz|z ggSy)zB Calculate the coefficient of lagrange polynomial rrrr rN)r)r{r lambda_list denominator1 denominator2 denominator3 denominator4s r lagrange_polynomial_coefficientz1SASolverScheduler.lagrange_polynomial_coefficientsa  $$$K(1,,,, A:C5L aZQ+a.89 ^O{1~ A'FG Q+a.89 ^O{1~ A'FG  aZ'N[^; AQ\]^Q_@_`L'N[^; AQ\]^Q_@_`L'N[^; AQ\]^Q_@_`L $!!n_{1~5EN[^3lB  $!!n_{1~5EN[^3lB  $!!n_{1~5EN[^3lB "aZQ+a.0q>KN24q>KN24  Q+a.0q>KN24q>KN24  Q+a.0q>KN24q>KN24  Q+a.0q>KN24q>KN24  $!!n_{1~5 AF,V#AQ7%a.;q>9:%a.;q>9:# # "!n_{1~5 AF,V  $!!n_{1~5 AF,V#AQ7%a.;q>9:%a.;q>9:# # "!n_{1~5 AF,V  $!!n_{1~5 AF,V#AQ7%a.;q>9:%a.;q>9:# # "!n_{1~5 AF,V  $!!n_{1~5 AF,V#AQ7%a.;q>9:%a.;q>9:# # "!n_{1~5 AF,V E-- +r"c |dvsJ|t|k(sJdg}|j|dz |}t|D]}}d} t|D]Z} |jr'| ||| |j |dz | z |||zz } 6| ||| |j |dz | z ||zz } \|j | t||k(sJd|S)N)rr rrz4the length of lambda list must be equal to the orderrrz3the length of coefficients does not match the order)rrr'rurrr() r{rrrrr coefficientslagrange_coefficientr. coefficientjs r get_coefficients_fnz%SASolverScheduler.get_coefficients_fnDs $$$K((`*``( #CCEAI{[u -AK5\ ??#7#:1#=@j@j A ~|SA$K #7#:1#=@j@j A ~|A$K      , -< E)`+``)r"noiserrc  t|dkDr|dn|jdd}|t|dkDr|d}n td|t|dkDr|d}n td|t|dkDr|d}n td |t|d kDr|d }n td | tdd d |j} |j |j dz|j |j } } |j| \} } |j| \} } tj| tj| z }tj| tj| z }tj|}||z }g}t|D]n}|j |z }|j|j |\}}tj|tj|z }|j|p|j|||||}|}|jrM|dk(rG|j |j dz }|j|\}}tj|tj|z }|dxxdtjd|dzz|zz|dzdz |d|dzzzdz tjd|dzz| zzd|dzzdzz z z||z z z cc<|dxxdtjd|dzz|zz|dzdz |d|dzzzdz tjd|dzz| zzd|dzzdzz z z||z z zcc<t|D]i}|jr<|d|dzz| ztj|dz |zz||z| |dz zz }K|d|dzz | z||z| |dz zz }k|jr;| tj dtjd|dzz|zz z|z}n7|| ztj tjd|zdz z|z}|jr,tj|dz |z| | z z|z|z|z}n| | z |z|z|z}|j#|j$}|S)ag One step for the SA-Predictor. Args: model_output (`torch.Tensor`): The direct output from the learned diffusion model at the current timestep. prev_timestep (`int`): The previous discrete timestep in the diffusion chain. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. order (`int`): The order of SA-Predictor at this timestep. Returns: `torch.Tensor`: The sample tensor at the previous timestep. r prev_timestepNrrr z.missing `noise` as a required keyword argumentr.missing `order` as a required keyword argumentr,missing `tau` as a required keyword argumentrzPassing `prev_timestep` is deprecated and has no effect as model output conversion is now handled via an internal counter `self.step_index`r6rrr&r rtrkrrr*ri zeros_liker'r(rrurrfrzr%) r{rrrrrrrrmodel_output_listrhsigma_s0rgalpha_s0rj lambda_s0 gradient_parthrr.sialpha_sisigma_si lambda_sigradient_coefficientsx temp_sigma temp_alpha_s temp_sigma_s temp_lambda_s noise_partx_ts r !stochastic_adams_bashforth_updatez3SASolverScheduler.stochastic_adams_bashforth_updateXs;6$'t9q=QfjjRV6W >4y1}a !RSS =4y1}Q !QRR =4y1}Q !QRR ;4y1}1g !OPP  $ ^  !.. KK!+ , KK ( 77@!99(C(99W% '(::IIh'%))H*== ((0 y  u *A1$B!%!=!=dkk"o!N Hh (+eii.AAI   y )  * !% 8 8 8U`be f  ?? "[[1)<= -1-I-I*-U* l % , 7%))L:Q Q %a(iiS!Vx 789!tax1CF #3a#7%))QaZUVTVDW:X#X^_beghbh^hmn]n"ooq!=02( &a(iiS!Vx 789!tax1CF #3a#7%))QaZUVTVDW:X#X^_beghbh^hmn]n"ooq!=02(u rAaZii#q& H 456,A./(!a%1 2 1sAv:!8;PQR;S!SVgjknojohpVq!qq  r ?? 5::a%))BaK!O2L.L#MMPUUJwEIIa!e4Dq4H)IIEQJ ??))c1fIM*g.@AAE UXbbCX%*]:ZGCffQWWo r"this_model_outputrw last_noise this_samplec . t|dkDr|dn|jdd} |t|dkDr|d}n td|t|dkDr|d}n td|t|dkDr|d}n td |t|d kDr|d }n td |t|d kDr|d }n td | tddd|j} |j |j |j |j dz } } |j| \} } |j| \}} tj| tj| z }tj|tj| z }tj|}||z }g}t|D]n}|j |z }|j|j |\}}tj|tj|z }|j|p| |gz}|j|||||}|}|jr|dk(r|dxxdtjd|dzz|zz|dz |d|dzzzdz tjd|dzz| zzd|dzzdz|zz z zz cc<|dxxdtjd|dzz|zz|dz |d|dzzzdz tjd|dzz| zzd|dzzdz|zz z zzcc<t|D]i}|jr<|d|dzz| ztj|dz |zz||z||dz zz }K|d|dzz | z||z||dz zz }k|jr;| tj dtjd|dzz|zz z|z}n7|| ztj tjd|zdz z|z}|jr,tj|dz |z| | z z|z|z|z}n| |z |z|z|z}|j#|j$}|S)a One step for the SA-Corrector. Args: this_model_output (`torch.Tensor`): The model outputs at `x_t`. this_timestep (`int`): The current timestep `t`. last_sample (`torch.Tensor`): The generated sample before the last predictor `x_{t-1}`. this_sample (`torch.Tensor`): The generated sample after the last predictor `x_{t}`. order (`int`): The order of SA-Corrector at this step. Returns: `torch.Tensor`: The corrected sample tensor at the current timestep. r this_timestepNrz4missing `last_sample` as a required keyword argumentr z3missing `last_noise` as a required keyword argumentrz4missing `this_sample` as a required keyword argumentrrrrzPassing `this_timestep` is deprecated and has no effect as model output conversion is now handled via an internal counter `self.step_index`r6rr)r{rrwrrrrrrrrrhrrgrrjrr r rr.r r r rmodel_prev_listrrrrs r stochastic_adams_moulton_updatez1SASolverScheduler.stochastic_adams_moulton_updates>$'t9q=QfjjRV6W  4y1}"1g  !WXX  4y1}!!W  !VWW  4y1}"1g  !WXX =4y1}Q !QRR ;4y1}1g !OPP  $ ^  !.. KK ( KK!+ , 77@!99(C(99W% '(::IIh'%))H*== ((5 y  u *A1$B!%!=!=dkk"o!N Hh (+eii.AAI   y )  * ,/@.AA $ 8 8 8U`be f  ?? &a(iiS!Vx 7891uQaZ 01 4uyy!c1f*RSQSAT7U U[\_bde_e[ejkZknoZoppr( &a(iiS!Vx 7891uQaZ 01 4uyy!c1f*RSQSAT7U U[\_bde_e[ejkZknoZoppr( u pAaZii#q& H 456,A./&Ah/ 0 1sAv:!8;PQR;S!SVehilmhmfnVo!oo  p ?? 5::a%))BaK!O2L.L#MMPZZJwEIIa!e4Dq4H)IIJVJ ??))c1fIM*g.@AAE UXbbCX%*]:ZGCffQWWo r"rschedule_timestepsc| |j}||k(j}t|dk(rt|jdz }|St|dkDr|dj}|S|dj}|S)a Find the index for a given timestep in the schedule. Args: timestep (`int` or `torch.Tensor`): The timestep for which to find the index. schedule_timesteps (`torch.Tensor`, *optional*): The timestep schedule to search in. If `None`, uses `self.timesteps`. Returns: `int`: The index of the timestep in the schedule. rr)rqnonzerorr)r{rr index_candidatesrs r index_for_timestepz$SASolverScheduler.index_for_timestep]s  %!% .(:CCE  A %T^^,q0J ! "Q &)!,113J*!,113Jr"c|jVt|tjr%|j |j j }|j||_y|j|_y)z Initialize the step_index counter for the scheduler. Args: timestep (`int` or `torch.Tensor`): The current timestep for which to initialize the step index. N) r isinstancer*Tensorrzrqrr$rxry)r{rs r _init_step_indexz"SASolverScheduler._init_step_indexsW    #(ELL1#;;t~~'<'<=#66x@D #00D r" return_dictc|j td|j|j||jdkDxr|jdu}|j ||}|rS|j |jd}|j||j|j||j|}tt|jj|jjdz dz D]@} |j | dz|j | <|j| dz|j| <B||j d<||jd<t#|j$||j&|j(} |jj*rt-|jjt/|j0|jz } t-|jjt/|j0|jz dz} n,|jj} |jj} t-| |j2dz|_t-| |j2d z|_ |j4dkDsJ|jdkDsJ||_| |_ |j |jd}|j7||| |j4| } |j2t|jj|jjdz kr|xj2dz c_|xj8dz c_|s| fSt;| S) a Predict the sample from the previous timestep by reversing the SDE. This function propagates the sample with the SA-Solver. Args: model_output (`torch.Tensor`): The direct output from learned diffusion model. timestep (`int`): The current discrete timestep in the diffusion chain. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. generator (`torch.Generator`, *optional*): A random number generator. return_dict (`bool`): Whether or not to return a [`~schedulers.scheduling_utils.SchedulerOutput`] or `tuple`. Returns: [`~schedulers.scheduling_utils.SchedulerOutput`] or `tuple`: If return_dict is `True`, [`~schedulers.scheduling_utils.SchedulerOutput`] is returned, otherwise a tuple is returned where the first element is the sample tensor. NzaNumber of inference steps is 'None', you need to run 'set_timesteps' after creating the schedulerrrrX)rrwrrrrr) generatorrr%r )rrrrr) prev_sample)rmr&rr(rwrrBrsrrthis_corrector_orderr'rrr^r?r@rtrrrr%rGr)rrqrvthis_predictor_orderrrxr)r{rrrr+r) use_correctormodel_output_convert current_taur.rr.r-r,s r stepzSASolverScheduler.steps0<  # # +s  ?? "  ! !( +!+L0@0@0L #88f8U --(:(:2(>?K99"6 ,,??"// :Fs4;;66 8S8SVW8WX[\\] >A$($6$6q1u$=D  q !$($6$6q1u$=D  q ! >"62!)2   &&$$   ;; ( (#&t{{'B'BCDWZ^ZiZiDi#j #&t{{'B'BCDWZ^ZiZiDilmDm#n #';;#>#> #';;#>#> $'(S>SVW>W$X!$'(S>SVW>W$X!((1,,,((1,,,!mmD$6$6r$:; <<-++ =   3t{{'B'BDKKD_D_bcDc#d d  ! !Q & ! A> !;77r"c|S)a? Ensures interchangeability with schedulers that need to scale the denoising model input depending on the current timestep. Args: sample (`torch.Tensor`): The input sample. Returns: `torch.Tensor`: A scaled input sample. r[)r{rrrs r scale_model_inputz#SASolverScheduler.scale_model_inputs  r"original_samplesrqc|jj|j|_|jj|j}|j|j}||dz}|j }t |j t |j kr=|jd}t |j t |j kr=d||z dz}|j }t |j t |j kr=|jd}t |j t |j kr=||z||zz}|S)a2 Add noise to the original samples according to the noise magnitude at each timestep (this is the forward diffusion process). Args: original_samples (`torch.Tensor`): The original samples to which noise will be added. noise (`torch.Tensor`): The noise to add to the samples. timesteps (`torch.IntTensor`): The timesteps indicating the noise level for each sample. Returns: `torch.Tensor`: The noisy samples. )rr$rSrXr)rerzrr%flattenrrr)r{r5rrqresqrt_alpha_prodsqrt_one_minus_alpha_prod noisy_sampless r add_noisezSASolverScheduler.add_noisesa2#1144).0es00r r3r3PsAF%>>qAFF>L E$("%BF  ('+",1"%/"&,116*/*/&)%*5\M'+ *1S, S,S, S,  S,  bjj$u+&= >? S,S,S,S,8$S,S,%*S, S,S, S, $D>!S,"!)#S,$"$%S,&"$'S,(UO)S,*"+S,, }-S,./S,01S,S,j  !!(3(O,O,U3PUP\P\K\E]O,d) ))X"J ,$ELL$RWR^R^$NTW\a\h\hFdg..rTs{$ BB A1,XX 5=*4 *4*4"/2*4 \\ *4Za/ a/r"