ia\:ddlZddlmZddlZddlmZddlmZddlmZddl m Z ddl m Z dd l mZmZdd lmZdd lmZdd lmZmZmZdd lmZddlmZmZmZmZddlm Z m!Z!ddl"m#Z#m$Z$m%Z%m&Z&ddl'm(Z(ddl)m*Z*Gdde*eZ+Gdde(Z,Gdde!Z-Gdde Z.Gddej^Z0GddeZ1Gd d!eZ2Gd"d#e$Z3Gd$d%e#Z4Gd&d'e&Z5ed()Gd*d+e5Z6Gd,d-e%Z7gd.Z8y)/N)Callable) Tokenizer)Unigram)nn)create_bidirectional_mask)BaseModelOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)TokenizersBackend)TransformersKwargsauto_docstringcan_return_tuple)check_model_inputs)LlamaAttentionLlamaRotaryEmbeddingapply_rotary_pos_embeager_attention_forward)ParakeetCTCConfigParakeetEncoderConfig)ParakeetEncoderBlock ParakeetEncoderConvolutionModuleParakeetForCTCParakeetPreTrainedModel)ParakeetProcessor) T5Tokenizerc deZdZ d fd Z d deeezdededzdedef dZxZ S) LasrTokenizerNc t |d|||||||d|tt|jdd|_y)N) eos_token unk_token pad_token extra_idsadditional_special_tokensvocab vocab_filerF)unk_id byte_fallback)super__init__rr _vocab_scores _tokenizer) selfr"r#r$r%r&r'r(kwargs __class__s o/home/obispo/Crisostomo_bridge/mision_env/lib/python3.12/site-packages/transformers/models/lasr/modular_lasr.pyr-zLasrTokenizer.__init__+s[  &?!  $ ""#   token_idsskip_special_tokensclean_up_tokenization_spaces group_tokensreturnc t|tr|g}|r%tj|Dcgc]}|d }}|Dcgc]}||jk7s|}}t j |f|||d|Scc}wcc}w)Nr)r5r6r7) isinstanceint itertoolsgroupby pad_token_idr _decode)r0r5r6r7r8r1 token_grouptokens r3r@zLasrTokenizer._decodeHs i %" I ;D;L;LY;WXKQXIX)2PuUd>O>O5OUP P ((   3)E     YQs A4A9A9)zzzdNNN)FNT) __name__ __module__ __qualname__r-r<listboolstrr@ __classcell__r2s@r3r r *sl"& @%*48!  c? " '+Tk      r4r c eZdZy) LasrProcessorNrDrErFr+r4r3rMrMasr4rMc\eZdZdZddddddddd d d d d dd d d dddddgddgddffd ZxZS)LasrEncoderConfiga This is the configuration class to store the configuration of a [`LasrEncoder`]. It is used to instantiate a `LasrEncoder` model according to the specified arguments, defining the model architecture. Configuration objects inherit from [`PreTrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PreTrainedConfig`] for more information. Args: hidden_size (`int`, *optional*, defaults to 512): Dimension of the layers and the hidden states. num_hidden_layers (`int`, *optional*, defaults to 17): Number of hidden layers in the Transformer encoder. num_attention_heads (`int`, *optional*, defaults to 8): Number of attention heads for each attention layer in the Transformer encoder. intermediate_size (`int`, *optional*, defaults to 2048): Dimension of the "intermediate" (often named feed-forward) layer in the Transformer encoder. hidden_act (`str` or `function`, *optional*, defaults to `"silu"`): The non-linear activation function (function or string) in the encoder and pooler. attention_bias (`bool`, *optional*, defaults to `False`): Whether to use bias in the attention layers. convolution_bias (`bool`, *optional*, defaults to `False`): Whether to use bias in convolutions of the conformer's convolution module. conv_kernel_size (`int`, *optional*, defaults to 32): The kernel size of the convolution layers in the Conformer block. subsampling_conv_channels (`int`, *optional*, defaults to 256): The number of channels in the subsampling convolution layers. subsampling_conv_kernel_size (`int`, *optional*, defaults to 5): The kernel size of the subsampling convolution layers. subsampling_conv_stride (`int`, *optional*, defaults to 2): The stride of the subsampling convolution layers. num_mel_bins (`int`, *optional*, defaults to 128): Number of mel features. dropout (`float`, *optional*, defaults to 0.1): The dropout ratio for all fully connected layers in the embeddings, encoder, and pooler. dropout_positions (`float`, *optional*, defaults to 0.0): The dropout ratio for the positions in the input sequence. layerdrop (`float`, *optional*, defaults to 0.1): The dropout ratio for the layers in the encoder. activation_dropout (`float`, *optional*, defaults to 0.1): The dropout ratio for activations inside the fully connected layer. attention_dropout (`float`, *optional*, defaults to 0.1): The dropout ratio for the attention layers. max_position_embeddings (`int`, *optional*, defaults to 10000): The maximum sequence length that this model might ever be used with. initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. layer_norm_eps (`float`, *optional*, defaults to 1e-06): The epsilon used by the layer normalization layers. feed_forward_residual_weights (`tuple[float, float]`, *optional*, defaults to `[1.5, 0.5]`): The residual weights for the feed forward layers. conv_residual_weights (`tuple[float, float]`, *optional*, defaults to `[2.0, 1.0]`): The residual weights for the convolution layers. batch_norm_momentum (`float`, *optional*, defaults to 0.01): The momentum for the batch normalization layers. rope_parameters (`RopeParameters`, *optional*): Dictionary containing the configuration parameters for the RoPE embeddings. The dictionary should contain a value for `rope_theta` and optionally parameters used for scaling in case you want to use RoPE with longer `max_position_embeddings`. Example: ```python >>> from transformers import LasrEncoderModel, LasrEncoderConfig >>> # Initializing a `LasrEncoder` configuration >>> configuration = LasrEncoderConfig() >>> # Initializing a model from the configuration >>> model = LasrEncoderModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` This configuration class is based on the LasrEncoder architecture from Google Health AI. You can find more details and pre-trained models at [TODO/TODO](https://huggingface.co/TODO/TODO). isiluF rg?i'g{Gz?gư>g?g?g@g?g{Gz?Nc ||_||_||_||_||_t |did|d|d|d|d|d|d|d|d | d | d | d | d | d|d|d|d|d|d|||`|`y)N hidden_sizenum_hidden_layersnum_attention_headsintermediate_size hidden_actattention_biasconvolution_biasconv_kernel_sizesubsampling_conv_channels num_mel_binssubsampling_conv_kernel_sizesubsampling_conv_stridedropoutdropout_positions layerdropactivation_dropoutattention_dropoutmax_position_embeddingsinitializer_ranger+) rope_parameterslayer_norm_epsfeed_forward_residual_weightsconv_residual_weightsbatch_norm_momentumr,r-subsampling_factor scale_input)r0r[r\r]r^r_r`rarbrcrerfrdrgrhrirjrkrlrmrorprqrrrnr1r2s r3r-zLasrEncoderConfig.__init__s8 /,-J*%:"#6   # / !4 0  "  *  . . '@ & *F %<  0   2! "0# $%<% &0) .  #  r4)rDrErF__doc__r-rJrKs@r3rPrPesfK^"%%& ! %'*Cj"Cj 3::r4rPcJeZdZdZ ddeezffd ZedZxZ S) LasrCTCConfiga This is the configuration class to store the configuration of a [`LasrForCTC`]. It is used to instantiate a Lasr CTC model according to the specified arguments, defining the model architecture. Configuration objects inherit from [`PreTrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PreTrainedConfig`] for more information. Args: vocab_size (`int`, *optional*, defaults to 512): Vocabulary size of the model. ctc_loss_reduction (`str`, *optional*, defaults to `"mean"`): Specifies the reduction to apply to the output of `torch.nn.CTCLoss`. Only relevant when training an instance of [`LasrForCTC`]. ctc_zero_infinity (`bool`, *optional*, defaults to `True`): Whether to zero infinite losses and the associated gradients of `torch.nn.CTCLoss`. Infinite losses mainly occur when the inputs are too short to be aligned to the targets. Only relevant when training an instance of [`LasrForCTC`]. encoder_config (`Union[dict, LasrEncoderConfig]`, *optional*): The config object or dictionary of the encoder. pad_token_id (`int`, *optional*, defaults to 0): Padding token id. Also used as blank token id. Example: ```python >>> from transformers import LasrForCTC, LasrCTCConfig >>> # Initializing a Lasr configuration >>> configuration = LasrCTCConfig() >>> # Initializing a model from the configuration >>> model = LasrForCTC(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` This configuration class is based on the Lasr CTC architecture from Google Health AI. You can find more details and pre-trained models at [TODO/TODO](https://huggingface.co/TODO/TODO). encoder_configc 0t|d|||||d|y)N) vocab_sizectc_loss_reductionctc_zero_infinityrxr?r+)r,r-)r0rzr{r|rxr?r1r2s r3r-zLasrCTCConfig.__init__s0  !1/)%    r4c4|jjdzS)Nr)rxrf)r0s r3inputs_to_logits_ratioz$LasrCTCConfig.inputs_to_logits_ratio$s""::A==r4)rQmeanTNr) rDrErFrudictrPr-propertyr~rJrKs@r3rwrwsCF!37  00  $>>r4rwc\eZdZdeffd Zdej dej fdZxZS)LasrEncoderSubsamplingconfigc(t|tj|j|j |_tj|j |j |j|j|_ tj|j |j|j|j|_ tj|j|j |_ tj|_y)N) kernel_sizestride)r,r-rLinearrdr[dense_0Conv1drerfconv_0rcconv_1dense_1ReLUact_fn)r0rr2s r3r-zLasrEncoderSubsampling.__init__*s yy!4!4f6H6HI ii      ;;11   ii     , ,;;11   yy!A!A6CUCUV ggi r4input_featuresr9c,|j|j|}|jdd}|j|j|}|j|j |}|jdd}|j |S)Nr)rr transposerrr)r0r hidden_statess r3forwardzLasrEncoderSubsampling.forward<sz DLL$@A %//15  DKK $>?  DKK $>? %//15 ||M**r4) rDrErFrPr-torchTensorrrJrKs@r3rr)s+ 0 $+ell+u||+r4rc eZdZy)LasrEncoderRotaryEmbeddingNrNr+r4r3rrEsr4rc eZdZdedeffd Z d dejdeejejfdzdejdzde e d eejejff d Z xZ S) LasrEncoderAttentionr layer_idxc4t|||d|_y)NF)r,r- is_causalr0rrr2s r3r-zLasrEncoderAttention.__init__Is +r4Nrposition_embeddingsattention_maskr1r9c |jdd}g|d|j}|j|j|j dd}|j |j|j dd}|j |j|j dd} |\} } t||| | \}}tj|jjt} | |||| |f|jsdn |j|jd|\} }| j g|dj#} |j%| } | |fS)NrrrY)rgscaling)shapehead_dimq_projviewrk_projv_projrr get_interfacer_attn_implementationrtrainingrkrreshape contiguouso_proj)r0rrrr1 input_shape hidden_shape query_states key_states value_statescossinattention_interface attn_output attn_weightss r3rzLasrEncoderAttention.forwardMsk$))#2. 88b8$--8 {{=166|DNNqRST [[/44\BLLQPQR {{=166|DNNqRST &S#7 jRUWZ#[ j(?(M(M KK , ,.E) %8      % $}}C$2H2HLL %  % ! \*k));;;;FFH kk+. L((r4NN) rDrErFrPr<r-rrtupler rrrJrKs@r3rrHs0SIM.2 ")||")#5<<#=>E") t+ ") +, ") u||U\\) * ")r4rc&eZdZddeffd ZxZS)LasrEncoderConvolutionModulerct|||d|_tj|j |j |_y)Nsame)momentum)r,r-paddingr BatchNorm1dr[rrnorm)r0r module_configr2s r3r-z%LasrEncoderConvolutionModule.__init__ss7 / NN6#5#5@Z@Z[ r4N)rDrErFrPr-rJrKs@r3rrrs\0\\r4rc eZdZdedeffd Z d dejdejdzdejdzdee d ejf d Z xZ S) LasrEncoderBlockrrcTt||||j|_|j|_t j |j |jd|_t j |j |jd|_ t j |j |jd|_ t j |j |jd|_ t j |j |jd|_ y)NF)bias) r,r-rprqr LayerNormr[ronorm_feed_forward1 norm_self_att norm_convnorm_feed_forward2norm_outrs r3r-zLasrEncoderBlock.__init__zs +-3-Q-Q*%+%A%A""$,,v/A/A6CXCX_d"e\\&*<*S>SZ_`f&8&8&:O:OV[\"$,,v/A/A6CXCX_d"e V%7%79N9NUZ[ r4Nrrrr1r9c 0|}|j|j|}|jd|z|jd|zz}|j|}|jd|||d|\}}||z}|j |j ||} |jd|z|jd| zz}|}|j|j|}|jd|z|jd|zz}|j|}|S)Nrr)rrr)rr+) feed_forward1rrpr self_attnconvrrq feed_forward2rr) r0rrrr1residualnormalized_hidden_statesr_ conv_outputs r3rzLasrEncoderBlock.forwardsJ!**4+B+B=+QR  . .q 1H >> from transformers import AutoProcessor, LasrEncoder >>> from datasets import load_dataset, Audio >>> model_id = TODO >>> processor = AutoProcessor.from_pretrained(model_id) >>> encoder = ParakeetEncoder.from_pretrained(model_id) >>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation") >>> ds = ds.cast_column("audio", Audio(sampling_rate=processor.feature_extractor.sampling_rate)) >>> inputs = processor(ds[0]["audio"]["array"]) >>> encoder_outputs = encoder(**inputs) >>> print(encoder_outputs.last_hidden_state.shape) ``` r)devicer)pr) target_length)r input_embedsrFT)rr)last_hidden_state)rrrarangerr unsqueezer functionalrgrrh_get_output_attention_maskrrrrandrirr ) r0rrr1rrr encoder_layerto_dropdropout_probabilitys r3rzLasrEncoder.forwards<7 ?? 5<< (;(;A(>}G[G[\ffghi S --mt||VZVcVc-d mm##C4+A+ADMM#Zmm##C4+A+ADMM#Z  %!<<^[h[n[nop[q<rN2;;&)  "[[ MG}}&+jjn#&7"G -!!#1),c ! !    m4 ??r4r)rDrErFrP__annotations__base_model_prefixr-rrrrrr rr rrJrKs@r3rrs !0"/3?@ ?@ t+?@+, ?@  ?@?@r4rceZdZfdZxZS) LasrForCTCc 8tjdi|S)a Example: ```python >>> from transformers import AutoProcessor, LasrForCTC >>> from datasets import load_dataset, Audio >>> model_id = TODO >>> processor = AutoProcessor.from_pretrained(model_id) >>> model = LasrForCTC.from_pretrained(model_id) >>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation") >>> ds = ds.cast_column("audio", Audio(sampling_rate=processor.feature_extractor.sampling_rate)) >>> inputs = processor(ds[0]["audio"]["array"], text=ds[0]["text"]) >>> predicted_ids = model.generate(**inputs) >>> transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True) >>> print(transcription) ``` r+)r,generate) super_kwargsr2s r3rzLasrForCTC.generates, uw/,//r4)rDrErFrrJrKs@r3rrs 00r4r)rrrrMrPrwr )9r=collections.abcrr tokenizersrtokenizers.modelsrr masking_utilsrmodeling_outputsr modeling_utilsr r processing_utilsr tokenization_utils_tokenizersr utilsrrr utils.genericrllama.modeling_llamarrrrparakeet.configuration_parakeetrrparakeet.modeling_parakeetrrrrparakeet.processing_parakeetrt5.tokenization_t5rr rMrPrwModulerrrrrrrr__all__r+r4r3rs$ %6/F&>II/vvV =,4 K!24 n % H-HV6>%6>r+RYY+8')T\#C\.+.b1& W@%W@  W@t004 r4