h SSKrSSKJr SSKJr SSKJrJr SSKr SSK r SSK J r SSK JrJr \ R "\5rSS\S\S \4S jjrSS \ R,S \ R,S \\S \ R,4SjjrSS \ R,S \ R,S\S \ R,4SjjrSS \ R,S \ R,S\S \ R,4Sjjr"SS5r"SS5rS S\\\4S\\\\ R>\ R>4S\\ R@S\\ RBS\\ RDS \ R,4 Sjjr#g)!N)Mapping)contextmanager)OptionalUnion)logging)is_torch_npu_availableis_torch_xpu_availablenestedsepreturnc\^^S[S[S[SS4UU4Sjjm0nT"USU5 U$)z>Flatten dictionary and concatenate nested keys with separator.nestprefixintor Nc>UR5HIup4TU;a[STSUS35e[U[5(aT"XAU-T-U5 MCXBX-'MK g)Nz separator 'z' not allowed to be in key '')items ValueError isinstancer)rrrkvrecurser s B/home/james-whalen/.local/lib/python3.13/site-packages/trl/core.pyrflatten_dict..recurse scJJLDAax ;se3OPQsRS!TUU!W%%A:+T2#$VZ !)dictstr)r r flatrs ` @r flatten_dictr sA%d%C%t%%% D FB KrvaluesmaskaxiscUb!X-RUS9URUS9- $X-R5UR5- $)z,Compute mean of tensor with a masked values.)r#)sum)r!r"r#s r masked_meanr&.sJ  """-d0CCC ""$txxz11runbiasedc[X5nX- n[US-U5nU(a,UR5nUS:Xa [S5eXfS- - nXW-nU$)z.Compute variance of tensor with masked values.rzThe sum of the mask is zero, which can happen when `mini_batch_size=1`;try increase the `mini_batch_size` or `gradient_accumulation_steps`)r&r%r)r!r"r'meancentered_valuesvariancemask_sumbessel_corrections r masked_varr06sh v $DmO?A-t4H88: q=V  %1 5/ Or shift_meanc[X5[X5pCX- [R"US-5-nU(dXS- nU$)z!Whiten values with masked values.g:0yE>)r&r0torchrsqrt)r!r"r1r+varwhiteneds r masked_whitenr7Is<F):f+C# S4Z!88H  Orc8\rSrSrSrS\S\4SjrS\4SjrSrg ) LengthSamplerRz Samples a length min_value max_valuec6[[X55UlgN)listranger!)selfr;r<s r__init__LengthSampler.__init__Ws567 rr cT[RRUR5$r>)nprandomchoicer!)rAs r__call__LengthSampler.__call__Zsyy ,,r)r!N) __name__ __module__ __qualname____firstlineno____doc__intrBrH__static_attributes__rrr9r9Rs&8#8#8-#-rr9c2\rSrSrSr\\S55rSrg) PPODecorators^Fc#r# Sv UR(Ga[5(aI[R"5 [R R 5 [R"5 g[5(aI[R"5 [RR 5 [R"5 g[RR5(aI[R"5 [RR 5 [R"5 ggg7fr>) optimize_device_cacher gccollectr3xpu empty_cachernpucuda is_available)clss rempty_device_cache PPODecorators.empty_device_cacheas   $ $ $%''  %%' '))  %%' ((**  &&( + %sD5D7rQN) rJrKrLrMrV classmethodrr_rPrQrrrSrS^s"!rrSshape generatordevicedtypelayoutc jUnUSnU=(d [RnU=(d [R"S5nUb[U[5(dURR OUSRR nXrR :wa.US:Xa(SnUS:wa[ RSUSUSUS35 O'XrR :waUS ;a[S US US 35e[U[5(a[U5S :XaUSn[U[5(a`SUS S-n[U5Vs/sHn[R"XUXSUS9PM n n[R"U SS9RU5n U $[R"XXSUS9RU5n U $s snf)zA helper function to create random tensors on the desired `device` with the desired `dtype`. When passing a list of generators, you can seed each batch size individually. If CPU generators are passed, the tensor is always created on the CPU. rcpuNmpszBThe passed generator was created on 'cpu' even though a tensor on zB was expected. Tensors will be created on 'cpu' and then moved to zl. Note that one can probably slightly speed up this function by passing a generator that was created on the z device.)r\rYzCannot generate a z! tensor from a generator of type .r*)r*)rcrdrerf)dim)r3stridedrdrr?typeloggerwarningrlenr@randncatto) rbrcrdrerf rand_device batch_sizegen_device_typeilatentss r randn_tensorrytsKqJ  $u}}F  *u||E*F7A)T7R7R)**//XabcXdXkXkXpXp kk )o.FKXY_X`aKKQ(Sggmfno  +?0R1&9Z[jZkklmn n)T""s9~':aL )T""uQRy :& & KK1k_e f&  ))G+..v6 N++ebhillmst N s>#F0)/r>)T)NNNN)$rWcollections.abcr contextlibrtypingrrnumpyrEr3 accelerater transformersrr get_loggerrJrnrrr Tensorboolr&r0r7r9rStupler? GeneratorrdrerfryrQrrrs #%" G   H %C$"2 2ELL22Z_ZfZf2u||5<<4SXS_S_&%,,ellX]XdXd - -0JN%)#'%) .  .d5??3U__DEF. U\\ ". EKK . U\\ " .  \\ .r