iV *ddlZddlmZddlmZmZmZmZmZddl Z ddl Z ddl Z ddl mZmZddlmZmZddlmZmZerddlZeGd d eZGd d ZGd dZ ddedededde j:fdZGddeeZy)N) dataclass)ListLiteralOptionalTupleUnion) ConfigMixinregister_to_config) BaseOutputis_scipy_available)KarrasDiffusionSchedulersSchedulerMixincXeZdZUdZej ed<dZeej ed<y)DPMSolverSDESchedulerOutputaq Output class for the scheduler's `step` function output. Args: prev_sample (`torch.Tensor` of shape `(batch_size, num_channels, height, width)` for images): Computed sample `(x_{t-1})` of previous timestep. `prev_sample` should be used as next model input in the denoising loop. pred_original_sample (`torch.Tensor` of shape `(batch_size, num_channels, height, width)` for images): The predicted denoised sample `(x_{0})` based on the model output from the current timestep. `pred_original_sample` can be used to preview progress or for guidance. prev_sampleNpred_original_sample) __name__ __module__ __qualname____doc__torchTensor__annotations__rrw/home/obispo/Crisostomo_bridge/mision_env/lib/python3.12/site-packages/diffusers/schedulers/scheduling_dpmsolver_sde.pyrr s' 37(5<<07rrc.eZdZdZddZedZdZy)BatchedBrownianTreezGA wrapper around torchsde.BrownianTree that enables batches of entropy.Nc |j||\}}|_|jdtj|}|%tj ddgj }d|_ t||jdk(sJ|d}|Dcgc]>}tj|||j|j|j|ddd @c}|_y#t$r |g}d|_YewxYwcc}w) Nw0rlTFgư>) t0t1sizedtypedeviceentropytol pool_size halfway_tree)sortsigngetr zeros_likerandintitembatchedlenshape TypeErrortorchsdeBrownianIntervalr'r(trees)selfxr$r%seedkwargsr"ss r__init__zBatchedBrownianTree.__init__6s IIb"-B ZZe..q1 2 <==Ir2779D  !t9 * **AB     % %XXhhyy!     !6D DL !  s0"C"AC;"C87C8c ||kr||dfS||dfS)Nrr)abs rr-zBatchedBrownianTree.sortQsE1ay11bz1rc |j||\}}}tj|jDcgc] }||| c}|j|zz}|j r|S|dScc}w)Nr)r-rstackr9r.r3)r:r$r%r.treews r__call__zBatchedBrownianTree.__call__UscyyR( B KK$**=$b"= >$))dBR SLLq*ad*>sA.N)rrrrr? staticmethodr-rHrrrr r 3s#Q 622+rr c$eZdZdZddfdZdZy)BrownianTreeNoiseSamplerasA noise sampler backed by a torchsde.BrownianTree. Args: x (Tensor): The tensor whose shape, device and dtype to use to generate random samples. sigma_min (float): The low end of the valid interval. sigma_max (float): The high end of the valid interval. seed (int or List[int]): The random seed. If a list of seeds is supplied instead of a single integer, then the noise sampler will use one BrownianTree per batch item, each with its own seed. transform (callable): A function that maps sigma to the sampler's internal timestep. Nc|SrIr)r;s rz!BrownianTreeNoiseSampler.jsqrc||_|jtj||jtj|}}t|||||_yrI) transformr as_tensorr rF)r:r; sigma_min sigma_maxr<rPr$r%s rr?z!BrownianTreeNoiseSampler.__init__jsI" :;T^^EOO\eLf=gB'2r48 rc|jtj||jtj|}}|j||||z j j z SrI)rPrrQrFabssqrt)r:sigma sigma_nextr$r%s rrHz!BrownianTreeNoiseSampler.__call__osY 67XbHc9dByyR BG==?#7#7#999r)rrrrr?rHrrrrLrL[s 6:[9 :rrLnum_diffusion_timestepsmax_betaalpha_transform_type)cosineexpreturnc $|dk(rd}n|dk(rd}ntd|g}t|D]<}||z }|dz|z }|jtd||||z z |>t j |tj S)aB Create a beta schedule that discretizes the given alpha_t_bar function, which defines the cumulative product of (1-beta) over time from t = [0,1]. Contains a function alpha_bar that takes an argument t and transforms it to the cumulative product of (1-beta) up to that part of the diffusion process. Args: num_diffusion_timesteps (`int`): The number of betas to produce. max_beta (`float`, defaults to `0.999`): The maximum beta to use; use values lower than 1 to avoid numerical instability. alpha_transform_type (`"cosine"` or `"exp"`, defaults to `"cosine"`): The type of noise schedule for `alpha_bar`. Choose from `cosine` or `exp`. Returns: `torch.Tensor`: The betas used by the scheduler to step the model outputs. r\cftj|dzdz tjzdz dzS)NgMb?gT㥛 ?r )mathcospits r alpha_bar_fnz)betas_for_alpha_bar..alpha_bar_fns-88QY%/$''9A=>!C Crr]c2tj|dzS)Ng()rar]rds rrfz)betas_for_alpha_bar..alpha_bar_fns88AI& &rz"Unsupported alpha_transform_type: rr') ValueErrorrangeappendminrtensorfloat32)rYrZr[rfbetasir%t2s rbetas_for_alpha_barrrus0x' D  & '=>R=STUU E * +M ( (!e. . S\"- R0@@@(KLM <<U]] 33rcBeZdZdZeDcgc]}|j c}}ZdZe d2de de de de de e ejee fd e d e ed e ed e ed e e de de fdZ d3de e ej(fde ej(de fdZde e ej(fddfdZedZedZedZd4de fdZdej(de e ej(fdej(fdZ d5de de e ej:fde e fdZd Zd!Z d"ej(dej(fd#Z!d"ej(de dej(fd$Z" d6d"ej(de d%e d&e dej(f d'Z#ed(Z$ d7d)e ej(ejfde e ej(fde ej(ejfd*ed+e de e%e&ff d,Z'd-ej(d.ej(d/ej(dej(fd0Z(d1Z)ycc}}w)8DPMSolverSDEScheduleru DPMSolverSDEScheduler implements the stochastic sampler from the [Elucidating the Design Space of Diffusion-Based Generative Models](https://huggingface.co/papers/2206.00364) paper. This model inherits from [`SchedulerMixin`] and [`ConfigMixin`]. Check the superclass documentation for the generic methods the library implements for all schedulers such as loading and saving. Args: num_train_timesteps (`int`, defaults to 1000): The number of diffusion steps to train the model. beta_start (`float`, defaults to 0.00085): The starting `beta` value of inference. beta_end (`float`, defaults to 0.012): The final `beta` value. beta_schedule (`str`, defaults to `"linear"`): The beta schedule, a mapping from a beta range to a sequence of betas for stepping the model. Choose from `linear` or `scaled_linear`. trained_betas (`np.ndarray`, *optional*): Pass an array of betas directly to the constructor to bypass `beta_start` and `beta_end`. prediction_type (`str`, defaults to `epsilon`, *optional*): Prediction type of the scheduler function; can be `epsilon` (predicts the noise of the diffusion process), `sample` (directly predicts the noisy sample`) or `v_prediction` (see section 2.4 of [Imagen Video](https://huggingface.co/papers/2210.02303) paper). use_karras_sigmas (`bool`, *optional*, defaults to `False`): Whether to use Karras sigmas for step sizes in the noise schedule during the sampling process. If `True`, the sigmas are determined according to a sequence of noise levels {σi}. use_exponential_sigmas (`bool`, *optional*, defaults to `False`): Whether to use exponential sigmas for step sizes in the noise schedule during the sampling process. use_beta_sigmas (`bool`, *optional*, defaults to `False`): Whether to use beta sigmas for step sizes in the noise schedule during the sampling process. Refer to [Beta Sampling is All You Need](https://huggingface.co/papers/2407.12173) for more information. noise_sampler_seed (`int`, *optional*, defaults to `None`): The random seed to use for the noise sampler. If `None`, a random seed is generated. timestep_spacing (`str`, defaults to `"linspace"`): The way the timesteps should be scaled. Refer to Table 2 of the [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://huggingface.co/papers/2305.08891) for more information. steps_offset (`int`, defaults to 0): An offset added to the inference steps, as required by some model families. r Nnum_train_timesteps beta_startbeta_end beta_schedule trained_betasprediction_typeuse_karras_sigmasuse_exponential_sigmasuse_beta_sigmasnoise_sampler_seedtimestep_spacing steps_offsetc |jjrts tdt |jj|jj |jj gdkDr td|+tj|tj|_ n|dk(r-tj|||tj|_ nk|dk(r6tj|dz|dz|tjdz|_ n0|d k(rt||_ nt|d |jd |jz |_tj"|j d |_|j'|d|||_d|_| |_d|_d|_|j0j3d|_y)Nz:Make sure to install scipy if you want to use beta sigmas.rznOnly one of `config.use_beta_sigmas`, `config.use_exponential_sigmas`, `config.use_karras_sigmas` can be used.rhlinear scaled_linear?r squaredcos_cap_v2z is not implemented for ?r)dimcpu)configr}r ImportErrorsumr|r{rirrmrnrolinspacerrNotImplementedError __class__alphascumprodalphas_cumprod set_timesteps noise_samplerr~ _step_index _begin_indexsigmasto) r:rurvrwrxryrzr{r|r}r~rrs rr?zDPMSolverSDEScheduler.__init__s ;; & &/A/CZ[ [  ++T[[-O-OQUQ\Q\QnQno pst tA   $m5==IDJ h & H>QY^YfYfgDJ o - C3H[chcpcpquvvDJ 1 1,-@ADJ%7OPTP^P^O_&`a aDJJ& #mmDKKQ? .6IJ!2!"4 kknnU+ rtimestepschedule_timestepsr^c| |j}||k(j}t|dkDrdnd}||jS)ak Find the index of a given timestep in the timestep schedule. Args: timestep (`float` or `torch.Tensor`): The timestep value to find in the schedule. schedule_timesteps (`torch.Tensor`, *optional*): The timestep schedule to search in. If `None`, uses `self.timesteps`. Returns: `int`: The index of the timestep in the schedule. For the very first step, returns the second index if multiple matches exist to avoid skipping a sigma when starting mid-schedule (e.g., for image-to-image). rr) timestepsnonzeror4r2)r:rrindicesposs rindex_for_timestepz(DPMSolverSDEScheduler.index_for_timestepsN"  %!% %1::< wR>R KNQ.367 rnum_inference_stepsr(c  ||_|xs|jj}|jjdk(r6t j d|dz |t dddj}n'|jjdk(rw||jz}t jd||zjdddjjt }||jjz }n|jjdk(r\||jz }t j|d| jjjt }|dz}n"t|jjd t jd|jz |jz d z}t j|}t j |t jdt#||}|jj$rF|j'| }t j|Dcgc]}|j)||c}}n|jj*rG|j-|| }t j|Dcgc]}|j)||c}}n\|jj.rF|j1|| }t j|Dcgc]}|j)||c}}|j3||} t j4|d ggjtj6}t9j:|j=|}t9j>|dd|ddjAd|ddg|_!t9j:|}t9j:| } t9j>|dd|ddjAdg}| |ddd<tE|jGdr'|j=|t8j6|_$n|j=||_$d|_%d|_&d|_'d|_(|jBj=d|_!d|_)ycc}wcc}wcc}w)a Sets the discrete timesteps used for the diffusion chain (to be run before inference). Args: num_inference_steps (`int`): The number of diffusion steps used when generating samples with a pre-trained model. device (`str` or `torch.device`, *optional*): The device to which the timesteps should be moved to. If `None`, the timesteps are not moved. rrrrhNrAleadingrzY is not supported. Please make sure to choose one of 'linspace', 'leading' or 'trailing'.r) in_sigmas)rrg)r(r mpsr)*rrrurnprfloatcopyarangeroundastyperriarrayrloginterpr4r{_convert_to_karras _sigma_to_tr|_convert_to_exponentialr}_convert_to_beta_second_order_timesteps concatenaternr from_numpyrcatrepeat_interleaverstr startswithrrrrrr) r:rr(rur step_ratior log_sigmasrWsecond_order_timestepss rrz#DPMSolverSDEScheduler.set_timestepshs$7 1TT[[5T5T ;; ' ': 5 A':Q'>@S[`abfdfbfgllnI [[ ) )Y 6,0H0HHJ1&9:ZGNNPQUSUQUV[[]ddejkI 11 1I [[ ) )Z 7,t/G/GGJ#6J;GNNPUUW^^_deI NI;;//01JK A 3 33t7J7JJsRSVVF^ 9bii3v;&?H ;; ( (,,v,>FSY!Z%$"2"25*"E!Z[I [[ / /11FXk1lFSY!Z%$"2"25*"E!Z[I [[ ( (**VQd*eFSY!Z%$"2"25*"E!Z[I!%!=!=fj!Q#077 C!!&),,F,;iiVAb\-K-KA-NPVWYWZP[ \] $$Y/ !&!1!12H!IIIy!}im.M.Ma.PQR 0 !$Q$ v; ! !% (&\\& \FDN&\\\8DN # kknnU+ !A"["["[sS+SS c d}d}d}||}tj|}|dd||zz}||} tj| D cgc]} |j| |c} } | Scc} w)Nc.tj| SrI)rr]_ts rsigma_fnz?DPMSolverSDEScheduler._second_order_timesteps..sigma_fns662#; rc.tj| SrI)rr_sigmas rt_fnz;DPMSolverSDEScheduler._second_order_timesteps..t_fnsFF6N? "rrrA)rdiffrr) r:rrrrmidpoint_ratiore delta_time t_proposed sig_proposedrWrs rrz-DPMSolverSDEScheduler._second_order_timestepssz  # LWWQZ sVj>99  + HH|\ed..ujA\] ]s A.ctjtj|d}||ddtjfz }tj|dk\dj dj |jddz }|dz}||}||}||z ||z z } tj | dd} d| z |z| |zz} | j|j} | S)a Convert sigma values to corresponding timestep values through interpolation. Args: sigma (`np.ndarray`): The sigma value(s) to convert to timestep(s). log_sigmas (`np.ndarray`): The logarithm of the sigma schedule used for interpolation. Returns: `np.ndarray`: The interpolated timestep value(s) corresponding to the input sigma(s). g|=Nr)axisr )rr) rrmaximumnewaxiscumsumargmaxclipr5reshape) r:rWr log_sigmadistslow_idxhigh_idxlowhighrGres rrz!DPMSolverSDEScheduler._sigma_to_tsFF2::eU34 Jq"**}55))UaZq188a8@EE*JZJZ[\J]`aJaEbQ;!(#9_t , GGAq! Ug H , IIekk "rrc|dj}|dj}d}tjdd|j}|d|z z}|d|z z}||||z zz|z}|S)a Construct the noise schedule as proposed in [Elucidating the Design Space of Diffusion-Based Generative Models](https://huggingface.co/papers/2206.00364). Args: in_sigmas (`torch.Tensor`): The input sigma values to be converted. Returns: `torch.Tensor`: The converted sigma values following the Karras noise schedule. rArg@r)r2rrr) r:rrRrSrhoramp min_inv_rho max_inv_rhors rrz(DPMSolverSDEScheduler._convert_to_karrass%R=--/ $Q<,,. {{1a!9!9:AG, AG,  k(A BBsJ rct|jdr|jj}nd}t|jdr|jj}nd}||n|dj }||n|dj }t j t jtj|tj||}|S)a Construct an exponential noise schedule. Args: in_sigmas (`torch.Tensor`): The input sigma values to be converted. num_inference_steps (`int`): The number of inference steps to generate the noise schedule for. Returns: `torch.Tensor`: The converted sigma values following an exponential schedule. rRNrSrAr) hasattrrrRrSr2rr]rrar)r:rrrRrSrs rrz-DPMSolverSDEScheduler._convert_to_exponentials" 4;; , --II 4;; , --II!*!6IIbM|jjSrI)negr]rs rrz,DPMSolverSDEScheduler.step..sigma_fns668<<> !rrc>|jjSrI)rrrs rrz(DPMSolverSDEScheduler.step..t_fns::<##% %rrrepsilon v_predictionr rz+prediction_type not implemented yet: samplezprediction_type given as z, must be one of `epsilon`, or `v_prediction`)rr)rrrrrlrrLr~rrrrrzrrirexpm1rrr)r:rrrrr min_sigma max_sigmarrrWrXrret_nextrrrr derivativedtr sigma_fromsigma_tosigma_up sigma_down ancestral_ts rstepzDPMSolverSDEScheduler.step[s< ?? "  ! !( +    %#';;t{{Q#?#C#C#Et{{GXyI!9&)YX\XoXo!pD  " "%,, " & &%,, &  $ $KK0ET__q%89JKK! 34ET__5JKj!16aZ n44  ;; & &) 3#'#<#<%(:BVK#)K,,F#F [[ ( (N :#'#<#<%(:BVK#/K<;PQ>TUCUZ]B]3]#^+q.1,-$ [[ ( (H 4%&ST T+DKK,G,G+HHtu  ? #775@Je#B :?2K((#!!J'H8hkZ]Xq[5P&QT^`aTa&afi%ijH"A+! 3;Jz*K#K08A;>&HKeg,L--K&(:(:8A;QWHX(Y\c(cfn(nnK(($ '/'7$# '+$ A$  +{Ymnnroriginal_samplesnoiserc|jj|j|j}|jjdk(rvt j |ra|jj|jt j}|j|jt j}n@|jj|j}|j|j}|j |Dcgc]}|j||}}nG|j|jg|jdz}n|jg|jdz}||j}t|jt|jkr=|jd}t|jt|jkr=|||zz} | Scc}w)am Add noise to the original samples according to the noise schedule at the specified timesteps. Args: original_samples (`torch.Tensor`): The original samples to which noise will be added. noise (`torch.Tensor`): The noise tensor to add to the original samples. timesteps (`torch.Tensor`): The timesteps at which to add noise, determining the noise level from the schedule. Returns: `torch.Tensor`: The noisy samples with added noise scaled according to the timestep schedule. )r(r'rrhrrA)rrr(r'typeris_floating_pointrrnrrrr5flattenr4 unsqueeze) r:rrrrrre step_indicesrW noisy_sampless r add_noisezDPMSolverSDEScheduler.add_noises,'7'>'>FVF\F\]  " " ' '5 0U5L5LY5W!%!2!23C3J3JRWR_R_!2!` ! %5%<%>qAFF>L E$(#%BF(,116*/,0 *,, ,,,, ,,  ,,  bjj$u+&= >? ,,,,$D>,,!),,"$,,%SM,,,,,,,,`bf#eU\\12#HPQVQ]Q]H^# #> 1uell/B)C 1 133  !!(3( u||+,  <,0-1 M" M"c5<<'(M"&c] M"^  "JELLU\\2TW\a\h\hFdg..r-s !88 A2G 8*8 8"%+%+P::85=*4 *4*4"/2*4 \\ *4Z^ /NK^ /r