i;+^ddlmZddlmZmZmZmZmZmZm Z ddl Z ddl m Z er e e ZeGddZeGdd Zed Gd d Zed GddZee eefe eeeeedfffZe eeeeedffZeee eeffZy)) dataclass) TYPE_CHECKINGDictListLiteralOptionalTupleUnionN) get_loggerceZdZUdZdZeeed<dZeeed<dZ e ed<dZ e ded <dZ eed <dZeed <dZej"ed <dZej&j(j*ed <dZej&j(j*ed<dZej&j(j*ed<dZej&j(j*ed<dZeed<dZeed<dZedeeeffdZedeeeffdZ dededej"dej&j(j*fdZ!y)ContextParallelConfiga Configuration for context parallelism. Args: ring_degree (`int`, *optional*, defaults to `1`): Number of devices to use for Ring Attention. Sequence is split across devices. Each device computes attention between its local Q and KV chunks passed sequentially around ring. Lower memory (only holds 1/N of KV at a time), overlaps compute with communication, but requires N iterations to see all tokens. Best for long sequences with limited memory/bandwidth. Number of devices to use for ring attention within a context parallel region. Must be a divisor of the total number of devices in the context parallel mesh. ulysses_degree (`int`, *optional*, defaults to `1`): Number of devices to use for Ulysses Attention. Sequence split is across devices. Each device computes local QKV, then all-gathers all KV chunks to compute full attention in one pass. Higher memory (stores all KV), requires high-bandwidth all-to-all communication, but lower latency. Best for moderate sequences with good interconnect bandwidth. convert_to_fp32 (`bool`, *optional*, defaults to `True`): Whether to convert output and LSE to float32 for ring attention numerical stability. rotate_method (`str`, *optional*, defaults to `"allgather"`): Method to use for rotating key/value states across devices in ring attention. Currently, only `"allgather"` is supported. N ring_degreeulysses_degreeTconvert_to_fp32 allgather)ralltoall rotate_method_rank _world_size_device_mesh_flattened_mesh _ring_mesh _ulysses_mesh_ring_local_rank_ulysses_local_rankc|jd|_|jd|_|jdk(r|jdk(r td|jdks|jdkr td|jdkDr|jdkDr td|jdk7rt d|jdy)NzfEither ring_degree or ulysses_degree must be greater than 1 in order to use context parallel inferencezF`ring_degree` and `ulysses_degree` must be greater than or equal to 1.znUnified Ulysses-Ring attention is not yet supported. Please set either `ring_degree` or `ulysses_degree` to 1.rz=Only rotate_method='allgather' is supported for now, but got .)rr ValueErrorrNotImplementedErrorselfs m/home/obispo/Crisostomo_bridge/mision_env/lib/python3.12/site-packages/diffusers/models/_modeling_parallel.py __post_init__z#ContextParallelConfig.__post_init__Qs    # D     &"#D    q T%8%8A%=x    a 4#6#6#:ef f   a D$7$7!$;A     ,%OPTPbPbOccde  -returnc2|j|jfSN)rrr#s r% mesh_shapez ContextParallelConfig.mesh_shapefs  $"5"566r'cy)z$Dimension names for the device mesh.)ringulyssesr#s r%mesh_dim_namesz$ContextParallelConfig.mesh_dim_namesjs#r'rank world_sizedevicemeshc ||_||_||_||_|j|j z|kDr)t d|j d|jd|d|jj|_|jd|_ |jd|_ |jj|_ |jj|_ y)NzThe product of `ring_degree` (z) and `ulysses_degree` (z") must not exceed the world size (z).r-r.)rrrrrrr!_flattenrrrget_local_rankrrr$r1r2r3r4s r%setupzContextParallelConfig.setupos %    !1!1 1J >01A1A0BBZ[_[n[nZopRS]R^^`a  $zz224**V,!ZZ 2 $ > > @#'#5#5#D#D#F r')"__name__ __module__ __qualname____doc__rrint__annotations__rrboolrrrrrtorchr3r distributed device_mesh DeviceMeshrrrrrr&propertyr r+strr0r9r/r'r%rr(sn."&K#%$(NHSM( OT 6AM723AE3K GU\\ 6:E5   ( ( 3 3:@DOU&&22==D;?J!!--88?>BM5$$00;;B c ##*7E#s(O77#c3h##G#G3G GEL]L]LiLiLtLtGr'rc "eZdZUdZdZeeed<dZe ed<dZ e ed<dZ e jed<dZe jj j"ed<ddd e d e d e jd ee jj j"fd Zy)ParallelConfigz Configuration for applying different parallelisms. Args: context_parallel_config (`ContextParallelConfig`, *optional*): Configuration for context parallelism. Ncontext_parallel_configrrrr)r4r1r2r3r4c||_||_||_||_|j|jj ||||yyr*)rrrrrIr9r8s r%r9zParallelConfig.setupsL %   ' ' 3  ( ( . .tZ N 4r')r:r;r<r=rIrrr?rr>rrrAr3rrBrCrDr9r/r'r%rHrHs@DX&;<CE3K GU\\ 6:E5   ( ( 3 3:DH O O O O u((44??@ Or'rHT)frozencDeZdZUdZeed<dZeeed<dZe ed<dZ y)ContextParallelInputa Configuration for splitting an input tensor across context parallel region. Args: split_dim (`int`): The dimension along which to split the tensor. expected_dims (`int`, *optional*): The expected number of dimensions of the tensor. If provided, a check will be performed to ensure that the tensor has the expected number of dimensions before splitting. split_output (`bool`, *optional*, defaults to `False`): Whether to split the output tensor of the layer along the given `split_dim` instead of the input tensor. This is useful for layers whose outputs should be split after it does some preprocessing on the inputs (ex: RoPE). split_dimN expected_dimsF split_outputcVd|jd|jd|jdS)NzContextParallelInput(split_dim=, expected_dims=z, split_output=))rNrOrPr#s r%__repr__zContextParallelInput.__repr__sK00@@PQUQcQcPddstxuFuFtGGHI Ir') r:r;r<r=r>r?rOrrPr@rTr/r'r%rMrMs, N#'M8C='L$Ir'rMc6eZdZUdZeed<dZeeed<dZy)ContextParallelOutputa Configuration for gathering an output tensor across context parallel region. Args: gather_dim (`int`): The dimension along which to gather the tensor. expected_dims (`int`, *optional*): The expected number of dimensions of the tensor. If provided, a check will be performed to ensure that the tensor has the expected number of dimensions before gathering. gather_dimNrOc<d|jd|jdS)Nz!ContextParallelOutput(gather_dim=rRrS)rWrOr#s r%rTzContextParallelOutput.__repr__s&24??2CCSTXTfTfSgghiir') r:r;r<r=r>r?rOrrTr/r'r%rVrVs" O#'M8C='jr'rV.) dataclassesrtypingrrrrrr r rAutilsr r:loggerrrHrMrVrFr>ContextParallelInputTypeContextParallelOutputTypeContextParallelModelPlanr/r'r%r`s/$"MMM  H  UGUG UGp OO O@ $III0 $jjj0  #s(OU/6J1KUSgilSlMmmnn "4 56>SUX>X8YY  U+CE^+^%_ _`r'