ho\SSKrSSKJr S/r"SS\RR 5rg)N) dynamically_quantize_per_channelWeightOnlyInt8QuantLinearc^\rSrSrSrU4SjrS\RS\R4Sjr\ S\RR4Sj5r S r U=r$) ra This class is a replacement for `torch.nn.Linear`. It implements a mixed dtype matrix multiplication using int8 symmetric per-channel weight quantization. The primary goal of this class is to leverage int8 quantization for weights to reduce the memory footprint and computational requirements while performing linear transformations. This can be particularly beneficial for deploying models in low latency environments Attributes: w_int8 (torch.Tensor): The quantized weights in int8 format. scales (torch.Tensor): The scaling factors for each channel to convert the quantized weights back to floating point format during the forward pass. c>URS5nURS5n[TU]"U0UD6 URSU5 URSU5 g)a Initializes the WeightOnlyInt8QuantLinear module. Args: *args: Variable length argument list for `torch.nn.Linear`. **kwargs: Arbitrary keyword arguments. Must include 'w_int8' (int8 quantized weights) and 'scales' (scaling factors). w_int8scalesN)popsuper__init__register_buffer)selfargskwargsr r __class__s Z/home/james-whalen/.local/lib/python3.13/site-packages/torchao/quantization/weight_only.pyr "WeightOnlyInt8QuantLinear.__init__sSH%H% $)&) Xv. Xv.xreturncHURSURS5n[R"X@RR UR 55UR-nUR"/URSSQSP76nURbXPR- nU$)a Performs the forward pass of the quantized linear layer, which consists of mixed dtype matrix multiplication using int8 symmetric per-channel weight quantization. Args: x (torch.Tensor): The input floating point tensor to the quantized linear layer. *args: Additional positional arguments. **kwargs: Additional keyword arguments. Returns: torch.Tensor: The output floating point tensor after the quantized matrix multiplication and rescale. N) viewshapetorchmmr todtyper reshapebias)rrrrx_viewys rforward!WeightOnlyInt8QuantLinear.forward-s~AGGBK( HHV[[^^AGG4 5 C II (qwws| (R ( 99 NArmodcURn[USS[R5up4nSupgU"UUURSLUR 5R 5US9nURUlURUlU?URUl[UR55Rn URU 5 U$)a Converts a `torch.nn.Linear` module to a `WeightOnlyInt8QuantLinear` module. This method performs the conversion by dynamically quantizing the weights of the original floating point linear layer to int8 format and creating a new `WeightOnlyInt8QuantLinear` instance with these quantized weights and the corresponding scaling factors. Args: mod (torch.nn.Linear): The original `torch.nn.Linear` module to convert. Returns: WeightOnlyInt8QuantLinear: The converted quantized linear module with int8 weights. i)r)N)r!r r ) weightrrint8r!t contiguous in_features out_featuresnext parametersdevicer) clsr&w_fp32r r _zpfake_in_featuresfake_out_featuresnew_mod device_to_uses r from_float$WeightOnlyInt8QuantLinear.from_floatBs> D#uzz /3+  %88:((*  "oo"// Nxx S^^-.55  =!r)__name__ __module__ __qualname____firstlineno____doc__r rTensorr$ classmethodnnLinearr:__static_attributes__ __classcell__)rs@rrrsK / 5<<*!UXX__!!r)rutilsr__all__rDrErr<rrrJs+ 3 & 'VVr