i5ddlZddlmZddlmZddlmZmZmZm Z m Z m Z m Z m Z mZmZmZmZmZmZmZmZmZmZmZmZmZddlmZddlmZdd lm Z dd l!m"Z"dd l#m$Z$dd l%m&Z&dd l'm(Z(ddl)m*Z*ddl+m,Z,ddl-m.Z.ddl/m0Z0ddl1m2Z2ddl3m4Z4ddl5m6Z6ddl7m8Z8ddl9m:Z:ddl;mZ>ddl?m@Z@ddlAmBZBddlCmDZDide"de&de(de4d ed!e<d"e>d#e2d$e,d%e6d&e8d'e*d(e.d)eBd*e$d+eDd,e@e0e e:d-ZEidede de d$e ded ed!ed"ed#ed&ed'e d(e d%ed)ed*e d+ed,eeeed-ZFejeHZIGd.d/ZJGd0d1ZKd2eLfd3ZMd4eLfd5ZNd6ZOy)7N) AutoConfig)logging) AqlmConfigAutoRoundConfig AwqConfigBitNetQuantConfigBitsAndBytesConfigCompressedTensorsConfig EetqConfigFbgemmFp8ConfigFineGrainedFP8Config FPQuantConfig GPTQConfig HiggsConfig HqqConfig Mxfp4ConfigQuantizationConfigMixinQuantizationMethod QuantoConfig QuarkConfig SpQRConfig TorchAoConfig VptqConfig) HfQuantizer)AqlmHfQuantizer)AutoRoundQuantizer) AwqQuantizer)BitNetHfQuantizer)Bnb4BitHfQuantizer)Bnb8BitHfQuantizer)CompressedTensorsHfQuantizer)EetqHfQuantizer)FbgemmFp8HfQuantizer)FineGrainedFP8HfQuantizer)FPQuantHfQuantizer)GptqHfQuantizer)HiggsHfQuantizer)HqqHfQuantizer)Mxfp4HfQuantizer)QuantoHfQuantizer)QuarkHfQuantizer)SpQRHfQuantizer)TorchAoHfQuantizer)VptqHfQuantizerawqbitsandbytes_4bitbitsandbytes_8bitgptqaqlmquantoquarkfp_quanteetqhiggshqqzcompressed-tensors fbgemm_fp8torchaobitnetvptqspqr)fp8z auto-roundmxfp4c6eZdZdZedefdZedZy)AutoQuantizationConfigz The Auto-HF quantization config class that takes care of automatically dispatching to the correct quantization config given a quantization config stored in a dictionary. quantization_config_dictc v|jd}|jdds|jddr*|jddrdnd}tj|z}n | td|tvr,td|d t t jt|}|j|S) N quant_method load_in_8bitF load_in_4bit_4bit_8bitThe model's quantization config from the arguments has no `quant_method` attribute. Make sure that the model has been correctly quantizedUnknown quantization type, got  - supported types are: ) getrBITS_AND_BYTES ValueError AUTO_QUANTIZATION_CONFIG_MAPPINGlistAUTO_QUANTIZER_MAPPINGkeys from_dict)clsrErGsuffix target_clss f/home/obispo/Crisostomo_bridge/mision_env/lib/python3.12/site-packages/transformers/quantizers/auto.pyrVz AutoQuantizationConfig.from_dictxs/33NC # ' ' >BZB^B^_motBu 8 < <^U SWY`F-<>>( # #r[rDcreZdZdZedeezfdZedZedeezdedzfdZ e dZ y) AutoHfQuantizerz The Auto-HF quantizer class that takes care of automatically instantiating to the correct `HfQuantizer` given the `QuantizationConfig`. r]c Dt|trtj|}|j}|t j k(r|jr|dz }n|dz }|tvr,td|dttjt|}||fi|S)NrKrJrMrN) isinstancerjrDrVrGrrPrHrTrQrSrU)rWr]rcrGrYs rZ from_configzAutoHfQuantizer.from_configs )4 0"8"B"BCV"W *77  -<< <"//' ' 5 51,@/44678:  ,L9 -888r[c Ptj|fi|}|j|S)N)rDr_ro)rWrbrcr]s rZr_zAutoHfQuantizer.from_pretraineds*4DDEbmflm233r[quantization_config_from_argsNc |d}nd}t|tr;t|trtj|}ntj|}|g|j j |j j k7r:td|j j d|j j dt|tttttttfrW|U|j}|jD]\}}t!||||dt#|j%dz }|dk7r-t|ttfst'j(||St*j-||S)z handles situations where both quantization_config from args and quantization_config from model config are present. zYou passed `quantization_config` or equivalent parameters to `from_pretrained` but the model you're loading already has a `quantization_config` attribute. The `quantization_config` from the model will be used.zThe model is quantized with z but you are passing a z| config. Please make sure to pass the same quantization config class to `from_pretrained` with different loading attributes.z"However, loading attributes (e.g. z]) will be overwritten with the one you passed to `from_pretrained`. The rest will be ignored.)rnrjrrVrD __class__rerQrrr r rrget_loading_attributesitemssetattrrSrUwarningswarnloggerinfo)rWr]rq warning_msgloading_attr_dictattrvals rZmerge_quantization_configsz*AutoHfQuantizer.merge_quantization_configss ) 4y  K )4 07I&5&?&?@S&T#&<&F&FGZ&[# * 5#--66:W:a:a:j:jj./B/L/L/U/U.VVmoLoVoVo_o_n``FF  ###+( .9 = T T V .446 8 c+T37 8 ?EVE[E[E]@^?_`}~ ~K " Z0CkSgEh%i MM+ &#" KK $""r[c ^|jdd}|jdds|jddr*|jddrdnd}tj|z}n | td|tvr8t j d|d ttjd yy ) NrGrHFrIrJrKrLrMrNz~. Hence, we will skip the quantization. To remove the warning, you can delete the quantization_config attribute in config.jsonT) rOrrPrQrRrzwarningrSrTrU)rErGrXs rZsupports_quant_methodz%AutoHfQuantizer.supports_quant_methods/33NDI # ' ' >BZB^B^_motBu 8 < <^U SWY`F-<.register_config_fnsH 5 5xx/CDE E#67HI I36(0 r[r^)rrs` rZregister_quantization_configrs r[namecfd}|S)zRegister a custom quantizer.c~tvrtddt|ts t d|t<|S)Nz Quantizer 'rz!Quantizer must extend HfQuantizer)rTrQrrr)rWrs rZregister_quantizer_fnz1register_quantizer..register_quantizer_fn$sG ) ){4&0DEF F#{+?@ @'*t$ r[r^)rrs` rZregister_quantizerr!s ! r[c"t|d}|r!tj|jsd}|s|Q|r&tj |j||_n||_tj |j|}nd}||j |||j|}|j|}|j|}t|jdds&|jj}t|d||d<|||fS)Nr]F) pre_quantized) device_map weights_only dequantizevaluequant) hasattrrlrr]rrovalidate_environmentupdate_device_mapupdate_tp_planupdate_ep_planr`rG)configr]rr user_agentr hf_quantizerrGs rZget_hf_quantizerr1sF$9:M_BB6C]C]^ +7 )8)S)S**,?*F &*=F &&22  & &'3  ))!% * "33J? ,,V4,,V4|77uM';;HHL"),"NJw   ++r[)Prxmodels.auto.configuration_autorutilsrutils.quantization_configrrrr r r r r rrrrrrrrrrrrrbaserquantizer_aqlmrquantizer_auto_roundr quantizer_awqrquantizer_bitnetr quantizer_bnb_4bitr!quantizer_bnb_8bitr"quantizer_compressed_tensorsr#quantizer_eetqr$quantizer_fbgemm_fp8r%quantizer_finegrained_fp8r&quantizer_fp_quantr'quantizer_gptqr(quantizer_higgsr) quantizer_hqqr*quantizer_mxfp4r+quantizer_quantor,quantizer_quarkr-quantizer_spqrr.quantizer_torchaor/quantizer_vptqr0rTrR get_loggerrerzrDrlstrrrrr^r[rZrs7.+4'/22F+6@2+-)-/-+1+ <++ O   O     " O  >6&!   O!" O#$ %$ ).$ 9$+$+$ J $  J $  J $ l$ [$ $ 9$1$/$ [$}$ $  J!$" J#$$ ! )$ .   H %&#&#Rssl   !S ! !,r[