i0ddlZddlmZddlmZddlZddlmcmZ ddlmZddl m Z ddl mZddlmZdd lmZmZmZmZdd lmZdd lmZdd lmZmZmZdd lmZm Z ddl!m"Z"ddl#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)ddl*m+Z+ddl,m-Z-ddl.m/Z/m0Z0m1Z1e'jde3Z4e%Gdde Z5ee%dGddeZ6ee%dGddeZ7ee%dGdd eZ8Gd!d"ejrZ:d#ejvd$ed-e>d.e"e$fd/Z?Gd0d1ejrZ@Gd2d3ejrZAGd4d5eZBGd6d7ejrZCe%Gd8d9e5ZDGd:d;ejrZEGd<d=ejrZFGd>d?ejrZGGd@dAejrZHGdBdCejrZIGdDdEejrZJGdFdGejrZKGdHdIejrZLGdJdKejrZMee%GdLdMeZNe%dNGdOdPe5ZOGdQdRejrZPGdSdTejrZQe%dUGdVdWe5ZRGdXdYe5eZSgdZZTy)\N)Callable) dataclass)nn)initialization)ACT2FN)Cache)%ClassifierFreeGuidanceLogitsProcessorGenerationMixinGenerationModeLogitsProcessorList)GenerateDecoderOnlyOutput)GradientCheckpointingLayer)BaseModelOutputBaseModelOutputWithPooling ModelOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)TransformersKwargsauto_docstringcan_return_tupleloggingtorch_compilable_check torch_int)check_model_inputs) AutoModel) JanusConfigJanusVisionConfigJanusVQVAEConfigcReZdZUeed<dZdZdZddgZddgZ dZ dZ dZ fd Z xZS) JanusPreTrainedModelconfigmodelimagetextTLlamaDecoderLayerJanusVisionEncoderLayerpast_key_values causal_maskct||t|trZt j |j tj|j jdjdyy)Nrr/) super _init_weights isinstanceJanusVisionEmbeddingsinitcopy_ position_idstorcharangeshapeexpand)selfmodule __class__s r/home/obispo/Crisostomo_bridge/mision_env/lib/python3.12/site-packages/transformers/models/janus/modeling_janus.pyr2z"JanusPreTrainedModel._init_weightsCs[ f% f3 4 JJv**ELL9L9L9R9RSU9V,W,^,^_f,g h 5)__name__ __module__ __qualname__r __annotations__base_model_prefixinput_modalitiessupports_gradient_checkpointing_no_split_modules_skip_keys_device_placement_supports_flash_attn_supports_sdpa_can_compile_fullgraphr2 __classcell__r>s@r?r$r$6sO (&*#,.GH#4m"DN!iir@r$z9 Base class for Janus VQ-VAE mode model outputs. ) custom_introcbeZdZUdZdZej dzed<dZej dzed<y)JanusVQVAEOutputz decoded_pixel_values (`torch.FloatTensor` of shape `(batch_size, num_channels, image_size, image_size)`): Reconstructed pixel values after encoding and decoding the input. embedding_loss (`torch.FloatTensor`): Embedding loss. Ndecoded_pixel_valuesembedding_loss) rArBrC__doc__rRr8 FloatTensorrDrSr@r?rQrQIs46:%++d29/3NE%%,3r@rQzy Base class for Janus model's outputs that may also contain a past key/values (to speed up sequential decoding). ceZdZUdZdZej dzed<dZe dzed<dZ e ej dzed<dZ e ej dzed<dZ e ej dzed<y)JanusBaseModelOutputWithPasta last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Sequence of hidden-states at the output of the last layer of the model. If `past_key_values` is used only the last hidden-state of the sequences of shape `(batch_size, 1, hidden_size)` is output. past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). Contains pre-computed hidden-states (key and values in the self-attention blocks and optionally if `config.is_encoder_decoder=True` in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. image_hidden_states (`tuple(torch.FloatTensor)`, *optional*): Tuple of `torch.FloatTensor` (one for the output of the image embeddings, `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder, and optionally by the perceiver Nlast_hidden_stater, hidden_states attentionsimage_hidden_states)rArBrCrTrYr8rUrDr,r rZtupler[r\rVr@r?rXrX[s|&37u((4/6$(OUT\(59M5**+d2926Je''(4/6;?u001D8?r@rXzQ Base class for Janus causal language model (or autoregressive) outputs. ceZdZUdZdZej dzed<dZej dzed<dZ e dzed<dZ e ej dzed<dZ e ej dzed<dZe ej dzed<y) JanusCausalLMOutputWithPastae loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Language modeling loss (for next-token prediction). logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`): Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax). past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. image_hidden_states (`tuple(torch.FloatTensor)`, *optional*): Tuple of `torch.FloatTensor` (one for the output of the image embeddings, `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder, and optionally by the perceiver Nlosslogitsr,rZr[r\)rArBrCrTr`r8rUrDrar,r rZr]r[r\rVr@r?r_r_|s"&*D%  d ")'+FE   $+$(OUT\(59M5**+d2926Je''(4/6;?u001D8?r@r_ceZdZdeffd Zdej dededej fdZd dej d e dej fd Z xZ S) r4r%cft|||_|j|_|j |_|j |_tj|j|j|j |j d|_ |j |j zdz|_ |j|_ tj|j|j|_|jdt!j"|jj%ddy)Nvalid) in_channels out_channels kernel_sizestridepaddingrr7r0F) persistent)r1__init__r% hidden_size embed_dim image_size patch_sizerConv2d num_channelspatch_embedding num_patches num_positions Embeddingposition_embeddingregister_bufferr8r9r;r<r%r>s r?rkzJanusVisionEmbeddings.__init__s  ++ ++ ++!yy++??  !OOt>1D!--"$,,t/A/A4>>"R ^U\\$:L:L-M-T-TU\-]jopr@ embeddingsheightwidthreturnc|jd}|jjjd}tjj s%||k(r ||k(r|j|j S|jjjd}|jd}||jz}||jz} t|dz} |jd| | |}|jdddd}tjj||| fdd }|jddddjdd|}|S) a This method allows to interpolate the pre-trained position encodings, to be able to use the model on higher resolution images. This method is also adapted to support torch.jit tracing and no class embeddings. Adapted from: - https://github.com/facebookresearch/dino/blob/de9ee3df6cf39fac952ab558447af1fa1365362a/vision_transformer.py#L174-L194, and - https://github.com/facebookresearch/dinov2/blob/e1277af2ba9496fbadf7aec6eba56e8d882d1e35/dinov2/models/vision_transformer.py#L179-L211 rrr/g?rrbicubicF)sizemode align_corners)r:rvweightr8jit is_tracingr7 unsqueezerorreshapepermuter functional interpolateview) r<ryrzr{rsrtpatch_pos_embeddim new_height new_widthsqrt_num_positionss r?interpolate_pos_encodingz.JanusVisionEmbeddings.interpolate_pos_encodingsE!&&q) //66<