hq :SSKJrJr SSKJrJr SSKrSSKrSSKJ r J r \"SS9"SS55r \ "5r S \ S \ 4S jr\ R4S \R S \ S\ S\R S\ S \R 4 Sjjr\ R4S\R S \ S\ S\R S\ S \R 4 SjjrS \ S\ S \R 4SjrS \ S \\\ \\ 44SjrS \ S\ S \R 4SjrS \ S \\R \R 44Sjrg)) dataclassfield)ListTupleN) GranularityPerAxisT)frozenc|\rSrSr%Sr\\S'Sr\\S'Sr\\S'\ "SS9r \ \\S '\ "S S9r \ \\S 'S r g )MarlinQQQConstantsTILE@ MIN_THREAD_N MAX_PARALLELcS/$)Nr_/home/james-whalen/.local/lib/python3.13/site-packages/torchao/quantization/marlin_qqq/utils.pyMarlinQQQConstants.s1#r)default_factorySUPPORTED_NUM_BITSc SS/$)Nrrrrrrsb#YrSUPPORTED_GROUP_SIZESrN)__name__ __module__ __qualname____firstlineno__rint__annotations__rrrrrr__static_attributes__rrrr r sHD#NL#L#$)+$FS F',=N'O49Orr num_bitsreturnc2SU-S:Xd SU35eSU-$)zCompute the packing factor for a given number of bits. Args: num_bits (int): Number of bits to pack. Returns: int: The packing factor. rzUnsupported num_bits = r)r&s rget_pack_factorr*s- =A C!8 CC  >rq_wsize_ksize_npermtilecURX4:XdeX-S:Xd SUSU35eX$-S:Xd SUSU35eURX-XBU-U45nURS5nURX-X$-45nURSUR545SS2U4RUR5nU$)avPermute weights to 16x64 Marlin tiles. Args: q_w (torch.Tensor): Quantized weights. size_k (int): Number of input features. size_n (int): Number of output features. perm (torch.Tensor): The computed permutation tensor to be applied. tile (int, optional): Tile size. Defaults to `TILE`. Returns: torch.Tensor: Weight tensor permuted to Marlin tiles. r size_k = , tile = rrN)shapereshapepermutenumel)r+r,r-r.r/s rmarlin_permute_weightsr;+s$ 99( (( ( =A B6()D6BB  =A B6()D6BB  ++v~tt^TB CC ++l #C ++v~v}5 6C ++r4::<( )!T' 2 : :399 EC Jr q_w_unpacked reverse_permcURSU4X-URSU-4:XdeX-S:Xd SUSU35eX$-S:Xd SUSU35eURSUR545SS2U4RUR5nURX-X$-XD45nURS5nURX45nU$)aReverse permute weights from 16x64 Marlin tiles. Args: q_w_unpacked (torch.Tensor): Unpacked quantized weights. size_k (int): Number of input features. size_n (int): Number of output features. reverse_perm (torch.Tensor): The computed reverse permutation tensor to be applied. tile (int, optional): Tile size. Defaults to `TILE`. Returns: torch.Tensor: Weight tensor reverse permuted from Marlin tiles. rr5r1r2rNr3)r7r8r:r9)r<r,r-r=r/q_w_comps rreverse_marlin_permute_weightsr@Ks $   q !6 *1%/   =A B6()D6BB  =A B6()D6BB ##R););)=$>? < gl  !  LMH -H 01H Or granularityc /n[S5Hn/nUS-nSHNnSUS--SUS--S-SUS--S-SUS--S-4H nURSU-U-SU--5 M" MP [S5H*nURUV s/sH oS U--PM sn 5 M, M [R"U5n US:XaG[ U[ 5(a[R"/S Q5n O3[R"/S Q5n O[S RU55eU RS [U 545SS2U 4R5n [R"U 5n U $s sn f)zPrecompute permutations for the marlin weight shuffling. Args: num_bits (int): Number of bits to pack. granularity (Granularity): The weight quantization granularity. Returns: torch.Tensor: The weight permutation tensor. r)r)rr5r5r4r6r )rrr5r4r6)rr4rrFr5r6rErGnum_bits must be 4, got {}rN)rangeappendextendnumpyarray isinstancer Exceptionformatr8lenraveltorch from_numpy) r&rA perm_listiperm1colblockrowjpr. interleaves rget_qqq_weight_permr^psaI 2Y1fEQU QU aQU aQU a   R#X^a%i78 qA   595a#'k59 : ;;y !D1} k7 + +%=>J%=>J4;;HEFF <<S_- .q*} = C C ED   D !D K:s E/ c \US:wa[SRU55e/n[S5H3nUR[S5Vs/sH o2SU--PM sn5 M5 /n[S5H+nURSVs/sH nSU-U-PM sn5 M- X4$s snfs snf)zPrecompute permutations for the marlin scale shuffling. Args: num_bits (int): Number of bits to pack. Returns: Tuple[List[int], List[int]]: Scale permutation list and scale permutation list for a single group. rrHrC)rr5rC r r4)rOrPrIrK)r& scale_permrVr[scale_perm_singles rget_qqq_scale_permsrfs1}4;;HEFFJ 1XeAh7hq1u9h78#% 1X  5Q!R5Q!a%!)5Q!RS  (( 8"Ss B$ B) c<[X5nUR5nU$)aReverse permutation for Marlin weight shuffling from `get_qqq_weight_perm`. Args: num_bits (int): Number of bits to pack. granularity (Granularity): The weight quantization granularity. Returns: torch.Tensor: The reversed weight permutation tensor. )r^argsort)r&rAr.s rget_qqq_weight_reverse_permris x 5D <<>D Krc[U5up[R"U5R5n[R"U5R5nX4$)a Reverse permutation for Marlin scale shuffling from `get_qqq_scale_perms`. Args: num_bits (int): Number of bits to pack. Returns: Tuple[List[int], List[int]]: The reversed scale permutation list and the reversed scale permutation list for a single group. )rfrStensorrh)r&rdres rget_qqq_scale_reverse_permsrlsJ%8$A!Jj)113J %67??A  ((r) dataclassesrrtypingrrrLrS torchao.quantization.granularityrrr constr#r*rTensorr;r@r^rfrirlrrrrrs )   $PPP  c c "     ,,     \\ J !,,! ! !,, !  !  \\ !J$#$K$ELL$N)#)%S 490D*E)&   +  \\  )# )% ell8R2S )r