hm0SSKJrJrJrJr SSKrSSKJr SSK J r J r SSK J r Sr\RR!5=(a \R"R$SLr\R(\R*\R,\R.1r\R2"5S!S\R4S\R6S \4S jj5r\R2"5SS\ R<S4S \R4S \S \ S\\S\R44 Sjj5r \R2"5SS\ R<SS4S\R4S\R6S \S \ S\\S \S\R44Sjj5r!S \R4S\R64Sjr"S \R4S\R4S\R44Sjr#S\R4S\R6S\\S44Sjr$Sr%S\S\S\4Sjr&S\R4S\\\\4S\R44Sjr'S\R44S jr(g)")IterableOptionalTupleUnionN)AsyncCollectiveTensor all_reduce)ScalingGranularityg-q=Famax float8_dtyperound_scales_to_power_of_2cLUR[R5nU[;aZ[R"U5R [R "U[S9- nUR[R5nO[SU35eU(a [U5nU$)zConverts the amax value of a tensor to the fp8 scale. Args: amax: The amax value of the tensor. float8_dtype: The float8 dtype. round_scales_to_power_of_2: if true, round scaling factor down to the nearest power of 2. )minUnsupported float8_dtype: ) totorchfloat64 FP8_TYPESfinfomaxclampEPSfloat32 ValueError_round_scale_down_to_power_of_2)r r r ress U/home/james-whalen/.local/lib/python3.13/site-packages/torchao/float8/float8_utils.py amax_to_scalersy 775== !Dy kk,'++ekk$C.HHffU]]#5l^DEE!-c2 Jx reduce_amaxscaling_granularity axiswise_dimreturncFU[RLa+[R"[R"U55nONU[R LdS5eUcS5e[R "[R"U5USS9nU(a[R"5(asUbUR5OSnUc'[[[R"555OUn[USU5n[U[5(aUR!5nU$)N unsupportedT)dimkeepdimMAX)r TENSORWISErrabsAXISWISEr distis_initialized get_grouplistrangeget_world_sizer isinstancerwait)rr device_meshr!r"r pggroups rtensor_to_amaxr78s0;;;yy1&"&8&A&AAP=PA'66'zz%))A,L$G t**,,(3(?[ " " $T68jU4..012b$u- d1 2 299;D Kr hp_tensorc4[UUUUU5n[XqUS9$)a Compute scaling factor for the given high precision tensor. Args: hp_tensor: high precision tensor float8_dtype: the float8 dtype to use reduce_amax: whether to reduce the max(abs(hp_tensor)) value across distributed ranks scaling_granularity: Defines the scaling granularity axiswise_dim: if axiswise granularity is used, defines the dim to scale across round_scales_to_power_of_2: if true, round scaling factor down to the nearest power of 2. )r )r7r)r8r r r4r!r"r r s rtensor_to_scaler:Us2*   D  7Q rcU[;aB[R"U5RnUR U*US9nUR U5$[ SU35e)aConverts a tensor to a saturated fp8 tensor. Note: The default behavior in PyTorch for casting to `float8_e4m3fn` and `e5m2` is to not saturate. In this context, we should saturate. A common case where we want to saturate is when the history of a tensor has a maximum value of `amax1`, and the current amax value is `amax2`, where `amax1 < amax2`. This is common when using delayed scaling. )rrr)rrrrrrr)rr max_values rto_fp8_saturatedr=vsVy KK -11 GG G 2ttL!!5l^DEEryc[RRU5n[RRX- 5nS[R"X#- 5-$)zComputes the error between two tensors in dB. For more details see: https://en.wikipedia.org/wiki/Signal-to-noise_ratio Args: x: The original tensor. y: The tensor to compare to the original tensor. )rlinalg vector_normlog10)rr>PsPns r compute_errorrFsD  ! !! $B  ! !!% (B  BG$ $$rtensor.clU[;a![R"U5RnO[ SU35eUR R URS9n[R"U5U:HR5R5nUS:HR5R5nXT4$)zCalculate FP8 tensor stats Args: tensor: The tensor to calculate stats for. float8_dtype: The float8 dtype. Returns: A tuple containing the number of zeros and the number of max values. r)dtyper) rrrrr_datar _orig_dtyper*sumitem)rGr FP8_MAXtensor_orig_typenum_maxnum_zeros rfp8_tensor_statisticsrRsy ++l+//5l^DEE||V-?-?@yy)*g5::<AACG A%**,113H  rcb[U5S:XdS5eUSUS:=(a USS:H$)Nz%is_row_major only supports 2D tensorsr)len)strides r is_row_majorrXs: v;! DDD  !9vay 3VAY!^3rsizealignment_valuecSUS- U--U-$)aY Returns the minimum alignment value that is greater than or equal to the given size. Args: size: The size of the data to be aligned. alignment_value: The alignment value to be used. Returns: int: The minimum alignment value that is greater than or equal to the given size. Usage: ``` >>> _get_min_alignment(10, 8) 16 ``` rU)rYrZs r_get_min_alignmentr]s" $(. /? BBrdimsc6UR5S:XdeURup#[U[5(aU4nSU;a [ US5OUnSU;a [ US5OUnXB- nXS- n[ R RRUSUSU45$)a Pads a 2D tensor with zeros to ensure that its dimensions are multiples of 16, which is required `torch._scaled_mm` Args: tensor: The tensor to pad. dims: Dimensions to pad. Returns: torch.Tensor: The padded tensor. Usage: ``` >>> pad_tensor_for_matmul(torch.randn((10, 10)), dims=0).shape torch.Size([16, 10]) >>> pad_tensor_for_matmul(torch.randn((10, 10)), dims=1).shape torch.Size([10, 16]) >>> pad_tensor_for_matmul(torch.randn((10, 10)), dims=(0, 1)).shape torch.Size([16, 16]) ``` rTrrU) r&shaper2intr]rnn functionalpad)rGr^dim1dim2 dim1_aligned dim2_alignedpad_dim1pad_dim2s rpad_tensor_for_matmulrls. ::<1  JD$w459%dB/$L349%dB/$L"H"H 88   " "6AxH+E FFrscalecUR[R:XdS5e[R"[R"[R "U555$)Nzscale must be float32 tensor)rIrrexp2floorlog2)rms rrrs? ;;%-- 'G)GG ' ::ekk%**U"34 55r)F))typingrrrrrtorch.distributed distributedr,)torch.distributed._functional_collectivesrrtorchao.float8.configr rcuda is_availableversionhipIS_ROCM float8_e4m3fn float8_e5m2float8_e4m3fnuzfloat8_e5m2fnuzrno_gradTensorrIboolrr)rbr7r:r=rFrRrXr]rlrr\rrrsz43 W4  ** ! ! # E (9(9(E       (- ,,++!%0.@.K.K"&  ||,  3-   \\ 8.@.K.K"&',||++ ,  3- !% \\@F FEKKF& %U\\ %ell %u|| % LL(-  38_,4 CSC3C3C(%G LL%G %c8C=&8 9%G \\%GP65<<6r