ȅihSSKrSSKrSSKJr SSKr/SQrSrSrSr Sr Sr S\R\\R-S S4S jr S\SS4S jrSS\SS4SS .S\\RS \R\\R-S-S\S\S\\R R"S-S\\RS-S S4SjjjrSS\\RS\S S4SjjrSS\\RS\\RS S4Sjjr\SS4S\\RS\\RS\S S4Sjjrg)N)Sequence) all_reducereduce broadcast all_gatherreduce_scatterc^[[RS5(d[R"SSS9 g[ 5nUHhnUR (a gUR5(d gUR(d gUR5nX1;a gURU5 Mj g)N_nccl_all_reducez)PyTorch is not compiled with NCCL support stacklevelFT) hasattrtorch_Cwarningswarnset is_sparse is_contiguousis_cuda get_deviceadd)tensorsdevicestensordevices I/home/james-whalen/.local/lib/python3.13/site-packages/torch/cuda/nccl.py is_availablers 588/ 0 0 AaPeG   ##%%~~""$   F c[RR5nUS- nUS- S-nUS-n[RR5R S5nUS:XaXU4$XX44$)a Returns the version of the NCCL. This function returns a tuple containing the major, minor, and patch version numbers of the NCCL. The suffix is also included in the tuple if a version suffix exists. Returns: tuple: The version information of the NCCL. izutf-8)rr _nccl_version_nccl_version_suffixdecode)vermajorminorpatchsuffixs rversionr,#sp (( "C 2IE BY% E %KE XX * * , 3 3G [RR5$N)rr_nccl_unique_idrr unique_idr18s 88 # # %%rcB[RRXU5$r.)rr_nccl_init_rank) num_ranksuidranks r init_rankr7<s 88 # #ID 99rinputsreturnc[U[RR5(a[U[R 5(a [ S5eg)Nz(Inputs should be a collection of tensors) isinstance collectionsabc ContainerrTensor TypeError)r8s r_check_sequence_typerA@sA fkoo77 8 8J ==BCC=rc|[U5 UcUn[U5 [RRXX#U5 gr.)rArrr r8outputsopstreamscommss rrrGs2 ! HHfrEBr)rDoutputrootrErFrDc[U5 Ub-Ub [S5e[R"S[SS9 XbnOp[ U[ R5(dH[ U[RR5(a[R"S[SS9 XnO UcXOUn[ RRXX#XE5 g)Nz'output' and 'outputs' can not be both specified. 'outputs' is deprecated in favor of 'output', taking in a single output tensor. The signature of reduce is: reduce(inputs, output=None, root=0, op=SUM, streams=None, comms=None).z`nccl.reduce` with an output tensor list is deprecated. Please specify a single output tensor with argument 'output' instead instead.r r z\nccl.reduce with an output tensor list is deprecated. Please specify a single output tensor.) rA ValueErrorrr FutureWarningr;rr?r<r=rr _nccl_reduce)r8rHrIrErFrGrD_outputs rrrQs   Y  MM`   mG  - -* ((33   5   ,"(.&,f HH&4WDrcZ[U5 [RRXX#5 gr.)rArr_nccl_broadcast)r8rIrFrGs rrr|s   HHV7:rcp[U5 [U5 [RRXX#5 gr.)rArr_nccl_all_gather)r8rDrFrGs rrrs(  ! HHfw>rcr[U5 [U5 [RRXX#U5 gr.)rArr_nccl_reduce_scatterrCs rrrs* ! HH!!&2Fr)rNN)NN)r<rcollections.abcr torch.cudar__all__SUMrr,r1r7r?rArintcudaStreamrrrrr0rrr\s$ P*-*&:D0F!FD4D $TC<@26  (E.2(E U\\ "(E LL8ELL1 1D 8(E (E (E ejj'' (4 / (Eell #d *(E (EXHL; U\\ ";*-; ;   ? U\\ "? ell #?  ?   G U\\ " G ell # G  G  Gr