i90dZddlmZddlmZddlmZddlZddlmZddl m Z dd l m Z dd lmZdd lmZdd lmZmZmZdd lmZmZddlmZddlmZmZmZmZm Z m!Z!ddl"m#Z#ddl$m%Z%m&Z&m'Z'e jPe)Z*dejVdejVfdZ,dejVdejVfdZ-dejVdejVfdZ.eedGddeZ/eedGdd eZ0eeGd!d"eZ1Gd#d$ejdZ3Gd%d&ejdZ4 dNd'ejdd(ejVd)ejVd*ejVd+ejVdzd,e5d-e5d.eefd/Z6Gd0d1ejdZ7Gd2d3ejdZ8Gd4d5eZ9eGd6d7eZ:Gd8d9ejdZ;Gd:d;ejdZ<ed<Gd=d>e:Z=Gd?d@ejdZ>edAGdBdCe:Z?eGdDdEe:Z@eGdFdGe:ZAeGdHdIe:ZBedJGdKdLe:ZCgdMZDy)OzPyTorch CLIP model.)Callable) dataclass)AnyN)nn)initialization)ACT2FN)create_causal_mask)GradientCheckpointingLayer)BaseModelOutputBaseModelOutputWithPoolingImageClassifierOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) ModelOutputTransformersKwargsauto_docstringcan_return_tuplelogging torch_int)check_model_inputs) CLIPConfigCLIPTextConfigCLIPVisionConfiglogitsreturnctjj|tjt ||j S)Ndevice)r functional cross_entropytorcharangelenr!)rs p/home/obispo/Crisostomo_bridge/mision_env/lib/python3.12/site-packages/transformers/models/clip/modeling_clip.pycontrastive_lossr(/s/ == & &vu||CKPVP]P]/^ __ similaritycZt|}t|j}||zdz S)Ng@)r(t)r* caption_loss image_losss r' clip_lossr/3s,#J/L!*,,.1J : % ,,r)tensorctj|d}tj|dd}tj|d}|S)z This method is equivalent to tensor.norm(p=2, dim=-1, keepdim=True) and used to make model `executorch` exportable. See issue https://github.com/pytorch/executorch/issues/3566 T)dimkeepdim?)r$powsum)r0 square_tensor sum_tensor normed_tensors r'_get_vector_normr<9s< IIfa(M=b$?JIIj#.M r)z} Base class for vision model's outputs that also contains image embeddings of the pooling of the last hidden states. ) custom_introceZdZUdZdZej dzed<dZej dzed<dZ e ej dfdzed<dZ e ej dfdzed<y)CLIPVisionModelOutputz image_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim)` *optional* returned when model is initialized with `with_projection=True`): The image embeddings obtained by applying the projection layer to the pooler_output. N image_embedslast_hidden_state. hidden_states attentions) __name__ __module__ __qualname____doc__r@r$ FloatTensor__annotations__rArBtuplerCr)r'r?r?Dsr .2L%##d*126u((4/6:>M5**C/047>7;Je'',-4;r)r?ze Base class for text model's outputs that also contains a pooling of the last hidden states. ceZdZUdZdZej dzed<dZej dzed<dZ e ej dfdzed<dZ e ej dfdzed<y)CLIPTextModelOutputz text_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim)` *optional* returned when model is initialized with `with_projection=True`): The text embeddings obtained by applying the projection layer to the pooler_output. N text_embedsrA.rBrC) rDrErFrGrNr$rHrIrArBrJrCrKr)r'rMrMVsr -1K""T)026u((4/6:>M5**C/047>7;Je'',-4;r)rMceZdZUdZdZej dzed<dZej dzed<dZ ej dzed<dZ ej dzed<dZ ej dzed<dZ e ed<dZe ed <d eefd Zy) CLIPOutputa loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `return_loss` is `True`): Contrastive loss for image-text similarity. logits_per_image (`torch.FloatTensor` of shape `(image_batch_size, text_batch_size)`): The scaled dot product scores between `image_embeds` and `text_embeds`. This represents the image-text similarity scores. logits_per_text (`torch.FloatTensor` of shape `(text_batch_size, image_batch_size)`): The scaled dot product scores between `text_embeds` and `image_embeds`. This represents the text-image similarity scores. text_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim`): The text embeddings obtained by applying the projection layer to the pooled output of [`CLIPTextModel`]. image_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim`): The image embeddings obtained by applying the projection layer to the pooled output of [`CLIPVisionModel`]. text_model_output (`BaseModelOutputWithPooling`): The output of the [`CLIPTextModel`]. vision_model_output (`BaseModelOutputWithPooling`): The output of the [`CLIPVisionModel`]. Nlosslogits_per_imagelogits_per_textrNr@text_model_outputvision_model_outputrcHtfdjDS)Nc3dK|]'}|dvr|nt|j)yw))rTrUN)getattrto_tuple).0kselfs r' z&CLIPOutput.to_tuple..s=  LLDGRYZ^`aRbRkRkRm m s-0)rJkeysr\s`r'rYzCLIPOutput.to_tuples# YY[   r))rDrErFrGrQr$rHrIrRrSrNr@rTr rUrJrrYrKr)r'rPrPhs&&*D%  d ")15e''$.504OU&&-4,0K""T)0-1L%##d*148186:3: %* r)rPceZdZdeffd Zdej dededej fdZd dejdej fd Z xZ S) CLIPVisionEmbeddingsconfigct|||_|j|_|j |_|j |_tjtj|j|_ tj|j|j|j |j d|_|j |j zdz|_|jdz|_tj"|j |j|_|j'dtj(|j j+ddy)NF) in_channels out_channels kernel_sizestridebiasr2r position_idsrr3 persistent)super__init__rb hidden_size embed_dim image_size patch_sizer Parameterr$randnclass_embeddingConv2d num_channelspatch_embedding num_patches num_positions Embeddingposition_embeddingregister_bufferr%expandr\rb __class__s r'rnzCLIPVisionEmbeddings.__init__s   ++ ++ ++!||EKK,GH!yy++??  !OOt>1D!--1"$,,t/A/A4>>"R ^U\\$:L:L-M-T-TU\-]jopr) embeddingsheightwidthrc|jddz }|jjjd}|jddz }tj j s%||k(r ||k(r|j|jS|ddddf}|ddddf}|jd} ||jz} ||jz} t|dz} |jd| | | }|jdddd}tjj|| | fdd }|jddddjdd| }t j ||fd S) a This method allows to interpolate the pre-trained position encodings, to be able to use the model on higher resolution images. This method is also adapted to support torch.jit tracing. Adapted from: - https://github.com/facebookresearch/dino/blob/de9ee3df6cf39fac952ab558447af1fa1365362a/vision_transformer.py#L174-L194, and - https://github.com/facebookresearch/dinov2/blob/e1277af2ba9496fbadf7aec6eba56e8d882d1e35/dinov2/models/vision_transformer.py#L179-L211 rrNr3r6rr2bicubicF)sizemode align_cornersr4)shaper|weight unsqueezer$jit is_tracingrirrrreshapepermuterr" interpolateviewcat) r\rrrryr|rzclass_pos_embedpatch_pos_embedr4 new_height new_widthsqrt_num_positionss r'interpolate_pos_encodingz-CLIPVisionEmbeddings.interpolate_pos_encodingsv!&&q)A- !44;;EEaH*003a7 yy##%+*F6UZ?**4+<+<= =,QU3,QU3r"t. T__, &}c'9:)11!5GI[]`a)11!Q1=--33 i( 4 *11!Q1=BB1b#Nyy/?;CCr) pixel_valuesc `|j\}}}}|sJ||jk7s||jk7r,td|d|d|jd|jd |jjj }|j|j |}|jdjdd}|jj|dd} tj| |gd } |r| |j| ||z} | S| |j|jz} | S) NzInput image size (*z) doesn't match model ().)dtyper2rr3r)rrq ValueErrorrxrrtoflatten transposerur~r$rrr|ri) r\rr batch_size_rr target_dtype patch_embeds class_embedsrs r'forwardzCLIPVisionEmbeddings.forwards6'3'9'9$ Avu'Vt-F%SWSbSbJb$VHAeW4KDOOK\\]^b^m^m]nnpq ++2288 ++LOO,O,OP #++A.88A> ++22:q"E YY l;C ##d&C&CJPVX]&^^J$d&=&=d>O>O&PPJr)F) rDrErFrrnr$TensorintrrHr __classcell__rs@r'raraseq/q,'D5<<'D'DUX'D]b]i]i'DRE$5$5Z_ZfZfr)rac eZdZdeffd Z d dej dzdej dzdejdzdejfdZ xZ S) CLIPTextEmbeddingsrbcNt||j}tj|j ||_tj|j||_|jdtj|jjddy)NrirjFrk) rmrnrorr{ vocab_sizetoken_embeddingmax_position_embeddingsr|r}r$r%r~r\rbrprs r'rnzCLIPTextEmbeddings.__init__s && !||F,=,=yI"$,,v/M/My"Y  ELL)G)GHOOPWXej  r)N input_idsri inputs_embedsrc8||jdn|jd}|jjjd}||kDrtd|d|||jddd|f}||j |}|j|}||z}|S)Nr3rzRSequence length must be less than max_position_embeddings (got `sequence length`: z and max_position_embeddings: )rr|rrrir)r\rrir seq_lengthmax_position_embeddingposition_embeddingsrs r'rzCLIPTextEmbeddings.forwards -6,AY__R(}GZGZ[]G^ !%!8!8!?!?!E!Ea!H . .d,<=S> )T^^ ;MdnnM]^NN#2' ]]D( // ii? ii? ii?  $..$..A r)NrBrrrc |j\}}}|j|}|j|}|j|} |j ||d|j j dd}|j ||d|j j dd}| j ||d|j j dd} tj|jjt} | |||| |f|j|jsdn |jd|\} } | j||dj!} |j#| } | | fS)z#Input shape: Batch x Time x Channelr3rr2)rr)rrrrrrrr get_interfacerb_attn_implementationrrrrrrr) r\rBrrrrrpqueriesr^valuesattention_interfacerrs r'rzCLIPAttention.forward4sT-:,?,?) J ++m,{{=)]+,,z:r4==ISSTUWXYyyZT]]CMMaQRSZRGQQRSUVW(?(M(M KK , ,.E) %8      % JJ#}}C$,, %  % ! \"))*j"EPPR mmK0 L((r)N)rDrErFrGrrrnr$rrrrJrrrs@r'rrstGB/.@B./3$)||$) t+$)+, $) u||U\\D00 1 $)r)rcVeZdZfdZdej dej fdZxZS)CLIPMLPct|||_t|j|_t j|j|j|_ t j|j|j|_ yr) rmrnrbr hidden_act activation_fnrrrointermediate_sizefc1fc2rs r'rnzCLIPMLP.__init__\sd  #F$5$5699V//1I1IJ99V55v7I7IJr)rBrcl|j|}|j|}|j|}|Sr)rrr)r\rBs r'rzCLIPMLP.forwardcs4/ **=9 / r))rDrErFrnr$rrrrs@r'rr[s$KU\\ellr)rceZdZdeezffd Zdejdejdee dejfdZ xZ S)CLIPEncoderLayerrbcDt||j|_t ||_t j|j|j|_ t||_ t j|j|j|_ yN)eps) rmrnrorpr self_attnr LayerNormlayer_norm_eps layer_norm1rmlp layer_norm2rs r'rnzCLIPEncoderLayer.__init__ksl ++&v.<<F> ')   q !=0  ((7 /  =0 r)) rDrErFrrrnr$rrrrHrrrs@r'rrjsVS/.@S|| +,     r)rcleZdZUeed<dZdZdZdZdZ dZ dZ e e dZej dZy)CLIPPreTrainedModelrbclip)imagetextT)rBrCc |jj}t|trt j |j jd|dzt j |jjd|dzt j|jtj|jjdjdnt|tr|jj}t j |j d|j"dz|zt j |j$j|jj&|zt j |jj|jj&|zt j|jtj|j(jdnt|t*r|jj}|j"dzd|jj,zdzz|z}|j"dz|z}t j |j.j|t j |j0j|t j |j2j|t j |j4j|nt|t6r|jj}|jj8dzd|jj,zdzz|z}d|jj8zdz|z}t j |j:j|t j |j<j|nt|t>rt j |j@j|jBdz|jjzt j |jDj|jFdz|jjznGt|tHrZt j |jDj|jj8dz|jjznt|tJrZt j |j@j|jj8dz|jjznst|tLrct j |jNj|jjPj8dz|jjzt|tRjTr>t jV|jXt jZ|jt|tRj\r-|jX t jV|jXy y y ) zInitialize the weightsrg{Gz?)meanstdr3rjr)rr2N)/rbinitializer_factor isinstancerinitnormal_rrr|copy_rir$r%rr~rarurprxinitializer_rangerzrnum_hidden_layersrrrrrrorr CLIPModeltext_projectiontext_embed_dimvisual_projectionvision_embed_dimCLIPVisionModelWithProjectionCLIPTextModelWithProjectionCLIPForImageClassification classifier vision_configrrzeros_rhones_r)r\rfactor in_proj_std out_proj_stdfc_stds r' _init_weightsz!CLIPPreTrainedModel._init_weightssu// f0 1 LL//66Sftm T LL2299RV W JJv**ELL9L9L9R9RSU9V,W,^,^_f,g h  4 5[[33F LL//cv?O?OQU?UX^?^ _ LL//66FMM<[<[^d LL((//KK++T1DKK4R4RR  ; < LL&&--KK++T1DKK4R4RR  : ; LL!!((KK--994?$++B`B``  fbll + KK $ JJv}} % fbii (V[[-D KK $.E (r)N)rDrErFrrIbase_model_prefixinput_modalitiessupports_gradient_checkpointing_supports_sdpa_supports_flash_attn_supports_flex_attn_supports_attention_backendrr_can_record_outputsr$no_gradrrKr)r'rrs[ (&*#N"&)# U]]_8%8%r)rc`eZdZdZdeffd Z d dejdzdee de fdZ xZ S) CLIPEncoderz Transformer encoder consisting of `config.num_hidden_layers` self attention layers. Each layer is a [`CLIPEncoderLayer`]. Args: config: CLIPConfig rbct|||_tjt |j Dcgc] }t|c}|_d|_ ycc}wNF) rmrnrbr ModuleListranger rlayersgradient_checkpointing)r\rbrrs r'rnzCLIPEncoder.__init__sO  mmuVMeMeGf$g!%5f%=$gh &+#%hsA#Nrrrc T|}|jD] }|||fi|}t|S)a7 Args: inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) )rA)r,r )r\rrrrB encoder_layers r'rzCLIPEncoder.forwardsH(& ![[ M)M +  r)r) rDrErFrGrrnr$rrrr rrrs@r'r'r'sK,z,/3  t+ +,    r)r'c eZdZdeffd Ze d dejdzdejdzdejdzdee de f d Z xZ S) CLIPTextTransformerrbct|||_|j}t ||_t ||_tj||j|_ |j|_ yr) rmrnrbrorrr'encoderrrrfinal_layer_norm eos_token_idrs r'rnzCLIPTextTransformer.__init__sa  && ,V4"6* " YF}G[G[\    ;%+74<<, '),  , ,== 112CD    !. .44Q7@Q@X@XY 5995F5M5M NUUZ\U]_M . .44Q7@Q@X@XYEII6G6N6NOSWSdSddB !M*/')77&11   r)r) rDrErFrrnrr$rrrr rrrs@r'r1r1s 0~ 0*..2,0 = <<$&=  t+= llT) = +, = $ = = r)r1zI The text model from CLIP without any head or projection on top. c eZdZUeed<dZddgZdeffd ZdejfdZ dZ e d e dd ejd zd ejd zdejd zdeedef dZxZS) CLIPTextModelrbrrrcdt||t||_|j yr)rmrnr1 text_model post_initrs r'rnzCLIPTextModel.__init__^s&  -f5 r)rcB|jjjSrrErrr_s r'get_input_embeddingsz"CLIPTextModel.get_input_embeddingsd))999r)c:||jj_yrrHr\rs r'set_input_embeddingsz"CLIPTextModel.set_input_embeddingsg5:""2r)Ftie_last_hidden_statesNrrrirc .|jd|||d|S)a9 Examples: ```python >>> from transformers import AutoTokenizer, CLIPTextModel >>> model = CLIPTextModel.from_pretrained("openai/clip-vit-base-patch32") >>> tokenizer = AutoTokenizer.from_pretrained("openai/clip-vit-base-patch32") >>> inputs = tokenizer(["a photo of a cat", "a photo of a dog"], padding=True, return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_state = outputs.last_hidden_state >>> pooled_output = outputs.pooler_output # pooled (EOS token) states ```rrrirK)rE)r\rrrirs r'rzCLIPTextModel.forwardjs/2t )%    r)r)rDrErFrrIr_no_split_modulesrnrModulerIrMrrr$rrrr rrrs@r'rBrBSs  -/AB~ :bii:;u5*..2,0  <<$&  t+ llT)  +,  $  6 r)rBc reZdZdeffd Ze d dejdzdedzde e de fdZ xZ S) CLIPVisionTransformerrbc t|||_|j}t ||_t j||j|_ t||_ t j||j|_ yr) rmrnrbrorarrrr pre_layrnormr'r3post_layernormrs r'rnzCLIPVisionTransformer.__init__sj  && .v6LL8M8MN"6*  ll9&:O:OPr)Nrrrrc $| td|j||}|j|}|jdd|i|}|j}|dddddf}|j |}t |||j|jS)Nz You have to specify pixel_values)rrrr:rK) rrrXr3rArYr rBrC)r\rrrrBr?rAr@s r'rzCLIPVisionTransformer.forwards  ?@ @ Ogh ))-8 +74<<, ', ,  ,==)!Q'2 ++M: )/')77&11   r)r))rDrErFrrnrr$rHboolrrr rrrs@r'rVrVsiQ/Q2605 ''$. #'+ +,  $   r)rVzK The vision model from CLIP without any head or projection on top. c eZdZUeed<dZdZdgZdeffd Zde jfdZ e d e ddejd zd ed eedefd ZxZS)CLIPVisionModelrbrrrcdt||t||_|j yr)rmrnrV vision_modelrFrs r'rnzCLIPVisionModel.__init__s'  1&9 r)rcB|jjjSrr`rrxr_s r'rIz$CLIPVisionModel.get_input_embeddings  ++;;;r)FrONrrc ,|jd||d|S)a( Example: ```python >>> from PIL import Image >>> import httpx >>> from io import BytesIO >>> from transformers import AutoProcessor, CLIPVisionModel >>> model = CLIPVisionModel.from_pretrained("openai/clip-vit-base-patch32") >>> processor = AutoProcessor.from_pretrained("openai/clip-vit-base-patch32") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> with httpx.stream("GET", url) as response: ... image = Image.open(BytesIO(response.read())) >>> inputs = processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_state = outputs.last_hidden_state >>> pooled_output = outputs.pooler_output # pooled CLS states ```rrrK)r`)r\rrrs r'rzCLIPVisionModel.forwards.>!t   %%=   r)r))rDrErFrrImain_input_namerrSrnrrTrIrrr$rHr[rrr rrrs@r'r]r]s $O!+,/ Q>QX]^<< T[[5W5W(XY r)Nrrrirrc x|jd|||dd|}|j}|j||_|S)a Examples: ```python >>> import torch >>> from transformers import AutoTokenizer, CLIPModel >>> model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32") >>> tokenizer = AutoTokenizer.from_pretrained("openai/clip-vit-base-patch32") >>> inputs = tokenizer(["a photo of a cat", "a photo of a dog"], padding=True, return_tensors="pt") >>> with torch.inference_mode(): ... text_features = model.get_text_features(**inputs) ```T)rrri return_dictrK)rEr;r )r\rrrir text_outputsr@s r'get_text_featureszCLIPModel.get_text_featuressV04C4??4 )% 4  4 %22 %)%9%9-%H "r)rrc v|jd||dd|}|j}|j||_|S)a Examples: ```python >>> import torch >>> from transformers import AutoProcessor, CLIPModel >>> from transformers.image_utils import load_image >>> model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32") >>> processor = AutoProcessor.from_pretrained("openai/clip-vit-base-patch32") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = load_image(url) >>> inputs = processor(images=image, return_tensors="pt") >>> with torch.inference_mode(): ... image_features = model.get_image_features(**inputs) ```T)rrrsrK)r`r;r)r\rrrvision_outputsr@s r'get_image_featureszCLIPModel.get_image_features:sU66GT5F5F6 %%=6  6  '44 '+'='=m'L$r) return_lossc L|jd||d|}|jd|||d|} |j} |j| } | j} |j | } | t | z } | t | z } t j| | jj| j} | |jjj| jz} | j} d}|r t| }t|| | | | | |S)a return_loss (`bool`, *optional*): Whether or not to return the contrastive loss. Examples: ```python >>> import torch >>> from transformers import AutoProcessor, CLIPModel >>> from transformers.image_utils import load_image >>> model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32") >>> processor = AutoProcessor.from_pretrained("openai/clip-vit-base-patch32") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = load_image(url) >>> inputs = processor( ... text=["a photo of a cat", "a photo of a dog"], images=image, return_tensors="pt", padding=True ... ) >>> with torch.inference_mode(): ... outputs = model(**inputs) >>> logits_per_image = outputs.logits_per_image # this is the image-text similarity score >>> probs = logits_per_image.softmax(dim=1) # we can take the softmax to get the label probabilities ```rerRN)rQrRrSrNr@rTrUrK)r`rEr;rr r<r$rr,rr!rqexpr/rP)r\rrrriryrrrwrtr@rNrSrRrQs r'rzCLIPModel.forward`sIL6GT5F5F6 %%=6 6  4C4??4 )%4  4 &33 --l; "00 **;7 $&6|&DD !$4[$AA  ,,{LNN4D4G4G HZHZ4[\)D,<,<,@,@,B,E,EkFXFX,YY*,,. _-D-+#%* .  r)NNr)NNNNNF)rDrErFrrIrSrnrrr$rrrrJr rurHr[rxrrPrrrs@r'r r s Z!z!F/3,0 <<  t+ llT)  +,  + +  D*/"''"#'"+, " + + ""H.215.204#').M ##d*M ''$.M  t+ M &&- M D[ M #'M +,M  M M r)r c eZdZUeed<dZddgZdeffd ZdejfdZ dZ e d e dd ejd zd ejd zdejd zdeedef dZxZS)rrbrCrrct||tj|}|j|_t j |j|jd|_ |jyNFrj) rmrnrBrorErrrornr rF)r\rbrErs r'rnz$CLIPTextModelWithProjection.__init__s[  "//7 $//!yy););V=R=RY^_ r)rcB|jjjSrrHr_s r'rIz0CLIPTextModelWithProjection.get_input_embeddingsrJr)c:||jj_yrrHrLs r'rMz0CLIPTextModelWithProjection.set_input_embeddingsrNr)FrONrrrirc |jd|||d|}|j}|j|}t||jS)a@ Examples: ```python >>> import torch >>> from transformers import AutoTokenizer, CLIPTextModelWithProjection >>> model = CLIPTextModelWithProjection.from_pretrained("openai/clip-vit-base-patch32") >>> tokenizer = AutoTokenizer.from_pretrained("openai/clip-vit-base-patch32") >>> inputs = tokenizer(["a photo of a cat", "a photo of a dog"], padding=True, return_tensors="pt") >>> with torch.inference_mode(): ... outputs = model(**inputs) >>> text_embeds = outputs.text_embeds ```rR)rNrArK)rEr;r rMrA)r\rrrirrtr@rNs r'rz#CLIPTextModelWithProjection.forwardsc44C4??4 )%4  4 %22 **=9 "#*<<  r)r)rDrErFrrIrrSrnrrTrIrMrrr$rrrrMrrrs@r'rrs  -/AB ~ :bii:;u5*..2,0 $ <<$&$  t+$ llT) $ +, $  $ 6$ r)rc eZdZUeed<dZdZdeffd ZdejfdZ e de d de jd zd ed eedefd ZxZS)rrbrr^ct||tj|}|j|_t j |j|jd|_ |jyr) rmrnr]ror`rrrornrrFr\rbr`rs r'rnz&CLIPVisionModelWithProjection.__init__s\  &33F; (55!#6+=+=v?T?T[`!a r)rcB|jjjSrrbr_s r'rIz2CLIPVisionModelWithProjection.get_input_embeddingsrcr)FrONrrc |jd||d|}|j}|j|}t||jS)a Examples: ```python >>> import torch >>> from transformers import AutoProcessor, CLIPVisionModelWithProjection >>> from transformers.image_utils import load_image >>> model = CLIPVisionModelWithProjection.from_pretrained("openai/clip-vit-base-patch32") >>> processor = AutoProcessor.from_pretrained("openai/clip-vit-base-patch32") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = load_image(url) >>> inputs = processor(images=image, return_tensors="pt") >>> with torch.inference_mode(): ... outputs = model(**inputs) >>> image_embeds = outputs.image_embeds ```re)r@rArK)r`r;rr?rA)r\rrrrwr@r@s r'rz%CLIPVisionModelWithProjection.forwardsb:6GT5F5F6 %%=6 6  '44 --m< $%,>>  r)r))rDrErFrrIrfrrnrrTrIrrr$rHr[rrr?rrrs@r'rrs $O! /