ȅi9SSKJr SSKrSSKJr SSKJr SSKJrJ r J r J r SSK r SSK JrJr SSKJrJr SSKJrJr SS KJr S S KJr \ (aSS KJrJr \ R:R=\S 5r \ R:R=\S5r!\"\ \ \#\ RH4r%\\"\/\%4r&\RN"SSS9"SS55r(\RN"SSS9"SS55r)\RN"SSS9"SS55r*S1Sjr+S2Sjr,S3Sjr-S4Sjr.S5Sjr/S6Sjr0S7Sjr1S8Sjr2S9Sjr3S9S jr4S:S!jr5\RN"S"S#55r6S;S$jr7SS)jr;\RN"SS*9"S+S,55r<\RN"SS*9"S-S.55r=S?S/jr>S@S0jr?g)A) annotationsN)Callable)Enum)AnyOptional TYPE_CHECKINGUnion)countersget_metrics_context)GraphPartitionMap InputType)get_plain_tensorsis_fake) OrderedSet)is_using_cudagraph_partition)SequenceSet cudagraphscudagraph_static_inputsT)frozenslotsc$\rSrSr%SrS\S'Srg) FunctionID z9Unique counter of a function wrapped in cudagraphify_implintidN__name__ __module__ __qualname____firstlineno____doc____annotations____static_attributes__rY/home/james-whalen/.local/lib/python3.13/site-packages/torch/_inductor/cudagraph_utils.pyrr s ? Gr'rcB\rSrSr%SrS\S'S\S'S\S'S\S 'S rg ) PlaceholderInfo'z A serializable version of torch.fx.Node that contains information pertinent to placeholder stack traces. We use these in logging and error messages related to cudagraphs, and will cache these results. strname Optional[str] stack_tracelist[PlaceholderInfo]usersmutating_use_stack_tracerNrrr'r(r*r*'s  I  ++r'r*cV\rSrSr%SrS\S'S\S'S\S'S \S 'S \S 'S\S 'Srg)WrappedFunction6z Represents a function that you want to record for CUDA graph replay, with a little more metadata so we can identify if we have an applicable CUDA graph in our CUDA graph tree for it. zCallable[..., Any]model Sequence[int]static_input_idxsrrztuple[torch.Tensor, ...] constantsSequence[PlaceholderInfo] placeholdersmutated_input_idxsrNrrr'r(r4r46s, $$N''++%%r'r4c[UR5S:Xa8[[UR55RR SS5$URHgnUR [RRRRLdM@URR SS5=n(dMeUs $ g)Nrr/) lenr1nextitermetagettargettorchopsatencopy_default)placeholder_nodeuser/s r(&get_mutating_use_stack_trace_from_noderKFs  ! !"a'D)//0166::=$OO%% ::--55 5!hhll=$??{?""& r'cUR$N)r2)placeholder_infos r(get_mutating_use_stack_tracerOUs  4 44r'cURnURRSS5n/nSnURS:Xa0URVs/sHn[ U5PM nn[ U5n[XX45$s snf)Nr/ placeholder)r-rArBopr1to_placeholder_inforKr*)rIr-r/r1r2is r(rSrSYs  D"''++M4@K E#m+1A1G1GH1GA$Q'1GH#I $   4e NN Is A9czURVs/sHoRS:XdM[U5PM! sn$s snf)NrQ)nodesrRrS)graphnodes r(get_placeholder_inforYhs9.3kk .9dWW =U!D!k  s88c SU3$)Nzskipping cudagraphs due to r)reasons r(format_default_skip_messager\ns ( 11r'cSnUHnXn[U5=n(dM O [S[U5S35nU(aUSU3$U$)Nzmutated inputs (z instances). Found from : )rOr\r>)r;mutation_indicesr/idxrQmsgs r(get_mutation_stack_tracercrsi"$K"' 6{C C; C  & 3/01= C( 66 Jr'c[RRRR(a?UR Vs/sH'nX0R ;aMU"X5(aM%UPM) nnO UR n[RSUR 5 [RSU5 U(a[URU5$S$s snf)Nz'check mutation static input indices: %sz#check mutation mutation indices: %s) rD _inductorconfigtritoncudagraph_treesr<r8static_inputs_logdebugrcr;)funcinputsis_cuda_graph_recorded_tensorrar`s r(check_for_mutationrns  $$44..+ .--- 1= . +  22143I3IACST  !!2!24DE!+ sC C*CcxURH*nURRSS5=n(dM(Us $ g)Nr/)r1rArB)rXrJr/s r(_get_use_stack_tracerps5zz((,,}d; ;; ;  r'cVUR[R"S5S5 [5(a&UR[R"S5S5 UR [R"S55=n(a=SUR S3n[ U5=n(a[USU35$[U5$[U5S:Xa1[[UR555RS:XagSU5n[S S RU535$) NrAcpuz cpu device ()r_rcudac38# UHn[U5v M g7frM)repr).0keys r( :check_multiple_devices_or_any_cpu_nodes..s:&9sc&9szmultiple devices: z, )poprDdevicerrBr-rpr\r>r?r@keystypejoin)device_node_mappingcpu_noderbr/ keys_reprs r('check_multiple_devices_or_any_cpu_nodesrsELL0$7$%% U 3T:&**5<<+>??x?X]]O1-.x8 8; 8.#6H /VW W*3//  A% )..01 2 7 76 A:&9:I &);DIIirYrWrc) gmmutated_inputsr<r8 default_msg unique_idxsrar` has_mutationr;s r(3check_for_mutation_ignore_cuda_graph_managed_tensorrs ..>?K $$44 !23 +=X+=CKAWC+=X+,1 +BHH5 ' GG>*a/ 't8[8Ys  B2B2cUR(a UR$URH"nUR(dMURs $ g)zE Gets the first non-empty stack trace of a placeholder or its users. N)r/r1)rQusers r(get_placeholder_stack_tracersF&&&!!    ## #" r'c.\rSrSrSrSrSrSrS SjrSr g) CheckInvariantStatusi rcURS:XagURS:XagURS:XagURSUR3$)NCudagraphManagedIdxMismatchz-cudagraph managed tensor data pointer changedStaticInputIdxMismatchz!static input data pointer changed&ExpectedDeadIndicesBeforeGraphMismatchz+expected dead indices before graph are livez: )r-r)rs r(__str__CheckInvariantStatus.__str__sK 995 5B YY2 26 YYB B@ii[4::,/ /r'rNrr,) r r!r"r#SUCCESSrrrrr&rr'r(rr s$G#$./*0r'rc [U5[U5:Xa[U5[U5:XdS5eUVs/sHoQUPM nnUVs/sHoRUPM nnUS3n[[Xg55Hyunup[U [R 5(deX5n U R 5U :wdMCX n USU RSU SU R 5S[U 5S3 nM{ U$s snfs snf)zq Logs the mismatch between input data pointers and recorded data pointers. This checks only idxs in target_idxs. zClength mismatch between inputs, recorded_data_ptr, and placeholdersz. z input name: z. data pointer changed from z to z. input stack trace:  ) r> enumerateziprrDTensordata_ptrr-r) r;rlrecorded_data_ptr target_idxsmismatchrT t_tensors t_data_ptrs error_msgtensorrindexrQs r(log_data_ptr_mismatchr!s v;#/0 0S[C DU5UM U%00KqKI01<=AQ'K=*C I!*3y+F!G F&%,,//// ??  (&-K+\+*:*:);<--5Jd6??;L:MN&&A+&N%OrS  "H 1=s C8C=cN^[UR55S-mSU4Sjjn[RRR R (aMT[RRR R :a[RU"55 gg)Nrc>STS3$)NzCUDAGraph supports dynamic shapes by recording a new graph for each distinct input size. Recording too many CUDAGraphs may lead to extra overhead. We have observed a0 distinct sizes. Please consider the following options for better performance: a) padding inputs to a few fixed number of shapes; or b) set torch._inductor.config.triton.cudagraph_skip_dynamic_graphs=True. Set torch._inductor.config.triton.cudagraph_dynamic_shape_warn_limit=None to silence this warning.r)num_cudagraphssr(warn_msg4maybe_warning_due_to_dynamic_shape..warn_msgFs 00>/?@' ' r'TFr) r>r}rDrerfrg"cudagraph_dynamic_shape_warn_limitrr)fn_cache new_int_keyrrs @r("maybe_warning_due_to_dynamic_shaper@st)A-N   %%HH  // ' ' J J K xz* r')rc8\rSrSr%SrS\S'S\S'S\S'S rg ) CudagraphCachedInfoi]z Info needed to realign inputs r:r;list[Optional[str]] stack_tracesz list[str]cudagraph_fail_reasonsrNrrr'r(rr]s,+%%%%r'rcL\rSrSr%SrS\S'S\S'S\S'S\S 'S \S 'S rg )CudagraphMetadataihz& Metadata for recording a CUDA graph. r:r;OrderedSet[int]r8r<rrzdict[str, torch.Tensor]r9rNrrr'r(rrhs',+&&''%%&&r'rcv/n[5n[5n[UR5HupVXaR;aUR U5 XaR ;aUR U5 UbUR UnO[SURSU3S/SS9nURU5 M /nURH7n U b URURU 5 M&URS5 M9 URV s0sHoURU _M n n [UUUUU 5$s sn f)z Convert the cudagraph metadata at the graph level to the graph partition level, given the graph partition info (i.e., mapping from partition input/output index to graph input/output index). N partition_ _placeholder_)r-r/r1r2)rrinput_index_mappingr8addr<r;r*rappendoutput_index_mappingrconstant_namesr9r) partition_mapmetadatapartition_placeholderspartition_static_input_idxspartition_mutated_input_idxspartition_input_idxgraph_input_idxrQpartition_stack_tracesgraph_output_idxr-partition_constantss r( get_partition_cudagraph_metadatarus` 3=<4>L 09))1, 88 8 ' + +,? @ 99 9 ( , ,-@ A  &"//@K*!-"2"2!3=AT@UV )- K %%k2'1* )>>  ' " ) )(*?*?@P*Q R " ) )$ / ?4A3O3O3O4h  &&3O #$   s D6c[U[R5(d [5$[5n[ U/S9H~n[ U5[RLaM![ U5(d+UR(dURR S:waM^URUR55 M U$![a Mf=f)zODebug helper that collects the data pointers of all CUDA tensors in the object.)outrt) rrDrrrr~ris_metar|rr Exception)objptrsbases r(collect_cuda_data_ptrsrs c5<< ( (|&LD!#2. :U\\ )  4==DLLDKK,<,<,F   HHT]]_ % / K   sC CC)rI torch.fx.Noderr.)rNr*rr.)rIrrr*)rWztorch.fx.Graphrr0)r[r,rr,)r;r:r`z&Union[AbstractSet[int], Sequence[int]]rr,)rkr4rllist[InputType]rmzCallable[[torch.Tensor], bool]rr.)rXrrr.)rz!dict[torch.device, torch.fx.Node]rr.)rbr,rr) rztorch.fx.GraphModulerzOrderedSet[str]r<rr8r7rr.)rQr*rr.) r;r:rlrrzSequence[Optional[int]]rr7rrrr,)rz)dict[tuple[int, ...], Callable[..., Any]]rrrbool)rr rrrr)robjectrr)@ __future__r dataclassescollections.abcrenumrtypingrrrr rDtorch._dynamo.utilsr r torch._inductor.utilsr r torch._subclasses.fake_tensorrrtorch.utils._ordered_setrutilsrrr AbstractSet_logginggetArtifactLoggerr rrilistrr OutputType ModelType dataclassrr*r4rKrOrSrYr\rcrnrprrrrrrrrrrrrrrr'r(rs"$66 =>D//<11(LINN44 ' (5ell!234 5 d9o& 2 3  d$/  0  d$/ , ,0 , d$/ & &0 & #  5 O 2+< (  $B >T:TT8H:HH J    99#9(9% 9  90 0400+ /  #   >7 : d#&&$& d# ' '$ '3$333lr'