ȅigN$SSKrSSKJrJr SSKJrJr SSKrSSK J r SSK J r SSK JrJr SSKJrJrJr /S Qr\ "S S 9S 5r\ "S S 9S 5r\ "S S 9\"SS555r\ "S S 9S S \R0R24S\R0R2S\\S\S\S\\R0R2S\\R0R2\\R0R2\\\4444 Sjj5r \ "S S 9SSj5r!g)N) dataclassfield)OptionalUnion) compatibility)map_arg) HolderModulelift_subgraph_as_module)CALLABLE_NODE_OPSis_node_output_tensorNodeList)getattr_recursivesetattr_recursive Component split_by_tags!move_non_tensor_nodes_on_boundaryF)is_backward_compatiblec(URS5H|n[U[RR5(a3[ US5(a X R ;aUR UnM] g[ X5(a [X5nM| g U$)N._modules)split isinstancetorchnn ModuleListhasattrrgetattr)objnamelayers U/home/james-whalen/.local/lib/python3.13/site-packages/torch/fx/passes/split_utils.pyrrsqC c588.. / /sJ''E\\,All5) S #%C! JcSU;a [XU5 gURS5n[[XS5SR USS5U5 g)Nrrr )setattrrrrjoin)rattrvaluer!s r"rr&sD $5! 3'#Qx0#((592EuMr#cj\rSrSr%Sr\R R\S'\ \S'\ \S'\ "\ S9r \ \S'\ "\ S9r\ \S'\ "\ S9r\ \S '\ "\S9r\\R R$\R R$4\S '\ "\ S9r\ \ \S 'S r\\R R,\S 'Srg )r/zP A component serves as a container for a subgraph we want to create afterwards. graphorderr )default_factoryinput_placeholders orig_inputs orig_outputs getattr_mapsconstructor_argsNgm)__name__ __module__ __qualname____firstlineno____doc__rfxGraph__annotations__intstrrlistr.r/r0dictr1Noder2r3r GraphModule__static_attributes__r4r#r"rr/s 88>> J I %T::d3K3t4L$48=T7RL$uxx}}ehhmm34R"'"=d3i=)-B%%&-r#rr3tagsreturn_fqn_mapping return_tupleGraphModuleClsreturnc  ^^^^S[RRRS[4Sjn0m0m0n/n0m[RR 5n0n Sn UHKn [ [RR 5[U5U 5mURT5 TXk'MM URRGHn U RS:XaU b [S5eU n M&U RS:XaOURU RU RS9X'[ R "U R"5XlMU RS :XaM[%U S 5(dS U R'535eU"U R(5U"U R*5-V s/sHn U RS ;dMTU PM nn XlR,mTTU '[/S U5SS9nTR0U:d$STRSTR0SU35eUUUU4SjnTRR3U U5nU R,UlUTU 'TTU'GM U c [S5eU"U R(S5H@n U RS :Xa(UR5U RU RS9X'M;STU 'MB TH3nURS:wdMTUR6RU5 M5 0nUGHrm[9[;TR<TR655nU(aTRR?U5 O/TRR?[U5S:XaUSOU5 [AUTRTRS9uTl!nUREU5 URGTR[9[;U R<TRH55SS9n[U5S:XaU(dUU TR6S'GM$[KTR65H5unn[RRMU5URU U'M7 GMu UR?[OU R(SU R<55 [QUVs0sHnURURB_M sn5nURRRUl)U"U R(S5H?n U RS :XdM[UUU R[WX RX55 MA U"UU5nU(aUU4$U$s sn fs snf)a Splits a GraphModule using tags on its graph nodes. We honor the order of tags. For example, we have tags = ["a", "b", "c"], the function will create the initial submodules in the order of "a", "b", "c". To set a tag: gm.graph.nodes[idx].tag = "mytag" This will result in all nodes with the same tag being extracted and placed in their own submodule. For placeholder, output and get_attr node, the tag is ignored. placeholder and output nodes are created when needed while get_attr nodes get copied to submodules where they are used. Given the following module def: class SimpleModule(torch.nn.Module): def __init__(self) -> None: super().__init__() self.linear1 = torch.nn.Linear(...) self.linear2 = torch.nn.Linear(...) self.linear3 = torch.nn.Linear(...) def forward(self, in1, in2): r1 = self.linear1(in1) r2 = self.linear2(in2) r3 = torch.cat([r1, r2]) return self.linear3(r3) Marking the node corresponding to in1 with the tag sc.REQUEST_ONLY.lower() results in the following split: ro: def forward(self, in1): self = self.root linear1 = self.linear1(in1) return linear1 main: def forward(self, in2, linear1): self = self.root linear2 = self.linear2(in2) cat_1 = torch.cat([linear1, linear2]) linear3 = self.linear3(cat_1) return linear3 main: def forward(self, in1, in2): self = self.root ro_0 = self.ro_0(in1) main_1 = self.main_1(in2, ro_0) return main_1 Returns: split_gm: torch fx graph after split orig_to_split_fqn_mapping: a map between the original fqn and the fqn after split for call_module and get_attr. xrHc4/n[XR5 U$)z3 Stores nodes in x to a list and returns the list. )rappend)rJrs r"flattensplit_by_tags..flattens88r#NoutputzMultiple output nodes in graph! placeholder type_exprget_attrtagzNode does not have tag: >rTrQc38# UHoRv M g7fN)r,).0cs r" split_by_tags..s7#6a''#6sr)defaultz Component z8 order must be >= max of its upstream components, order=z and max=c>URS:XaUTR;amTRRURUR S9TRU'[ R "UR5TRUlTRU$URS:waTUT:XaTU$UTR;aTRRU5 TRRURUR S9n[ R "UR5UlTRRU5 STU'TRTRRU5$)NrTrRrQ)opr1r+rTtargettypecopymetar/rLrQr r.index)rJrQcompnode_remappingnode_to_component used_in_mains r" remap_func!split_by_tags..remap_funcsBttz!D---+/::+>+>AFF,?,D%%a(15 !&&0AD%%a(-((++ tt}$):1)=)E%a(((((  ''*"jj44QVVqvv4N #'99QVV#4  ''..{;"& Q**4+;+;+A+A!+DE Er#zGraph had no output node!r )subgraph comp_name)argskwargs)-rr:nodeArgumentrr;rlenrLr+nodesr^ RuntimeErrorrQr r`rarbr format_noderlrmrUmaxr, node_copyrTr0tuplemap __getitem__rPr r3update call_moduler/ enumerateProxyrr _codegenr%rr_)r3rDrErFrGrNtag_to_componentall_componentsmain_gmain_remapping output_noderUrnrJupstream_componentsmxrhnorig_to_split_fqn_mappingoutscomp_orig_to_split_fqn_mapping main_nodeiord main_root result_gmrerfrgs ` @@@r"rrIs B588==))h:3%Id# $  77h &"#DEEK  77m ##)#5#5dii499#5#UN (, $))(> >tt66 ! a >   )"&$7#67 CzzR  #[\`\f\f[ggpqspt u   F F8 JJ z 2 t#!OR677 [%%a( ) 44: !'!&& IN #LO* 44= a - - 4 4Q 7 13S33T5F5FGH  JJ  d # JJ  Tad1gT B2I tyy3 // "(()GH&& IIs>55t7G7GHI' t9>,3 MM'+**1-~/I/IJK^L^Tdii0^LMIhh''FO[%%a( ) 44:  Iqvv'8XX'F G*y&1I333 m RMs.U5 U5!U:cp^^0m[U5HupURHnUTU'M M S[RRS[ [RR4U4SjjmS[RRS[ [RR4U4SjjnS[RRS[ S[4UU4SjjnS[RRS[ S[ S[[[[RR44UU4S jjn[U5GHduprUR(dM/n[5n URH5n[U5(aMU"X75(dM$URU5 M7 U(dMsURS 5n X;aM!U RU 5 U T;d TU U:waMCT"U 5n [U 5S :dS 5e[5n U H n TU nX:wdMU RU5 M" [U 5S :wa[!S U R"S35 MU R5nU"XU5unnU(GaUHnURR%U5 XRRU5 UTU'[!SU R"SUR"SUR(aSOSSUSXR(aSOSSU3 5 M U"U 5nUHdn[U5(aMUU ;dMUT;dM%TUU:XdM0U"UU5(dSUR"S35eURU5 Mf U(aGMGMg g)a Move non-tensor nodes on the boundary between subgraphs. For each subgraph: 1. Find nodes whose type is not tensor and any of its children is in another subgraph, put them in a queue for next step 2. Do a BFS on those nodes in the queue, and run a DFS for each node, let's say node X and it is in subgraph A: a. if it is in to_subgraph, return (continue DFS) b. if it is in from_subgraph, collect the nodes to nodes_to_move, and continue DFS c. otherwise, this means it cannot be moved d. also check if node X's parent should be put into the queue. (The queue may have duplicated nodes, just process the node once) Args: subgraphs: List of subgraphs containing nodes to be processed rnrHc>URVs/sH#nUR[;dMUT;dM!UPM% sn$s snf)z@Get children nodes that are in callable ops and in some subgraph)usersr^r )rnusernode_to_subgraphs r"get_children_in_graph@move_non_tensor_nodes_on_boundary..get_children_in_graph\sI  "ww++ 048H0H "   ===c>URVs/sH#nUR[;dMUT;dM!UPM% sn$s snf)z>Get parent nodes that are in callable ops and in some subgraph)all_input_nodesr^r )rnargrs r"get_parents_in_graph?move_non_tensor_nodes_on_boundary..get_parents_in_graphdsK++ +vv** /26F/F +   rcurrent_subgraph_idxcB>^T"U5n[UU4SjU55$)z Check if the node has any children in a subgraph different from current_subgraph_idx. This is the requirement used in both step 1 and step d. c34># UH nTUT:gv M g7frWr4)rXchildrrs r"rZ\move_non_tensor_nodes_on_boundary..has_children_in_other_subgraph..ts  IQ U #'; ;s)any)rnrchildrenrrs ` r"has_children_in_other_subgraphImove_non_tensor_nodes_on_boundary..has_children_in_other_subgraphls)). IQ   r# from_subgraph to_subgraphcn>^^^^^^[5m[5mSmUUUUUUUU4SjmT"U5 TT4$)a] Check if node and its dependencies can be moved from from_subgraph to to_subgraph. Returns (can_move, nodes_to_move) For node X, do a DFS on its descendants, for each node: - if it is in to_subgraph, return (continue DFS) - if it is in from_subgraph, collect the nodes to nodes_to_move, and continue DFS - otherwise, this means it cannot be moved Tc>UT ;agT RU5 UT;agTUnUT :XagUT:XaT RU5 OSmgT"U5nUHnT(dM T"U5 M g)NF)add) current_nodecurrent_subgraphrrcan_movedfsrrr nodes_to_movervisiteds r"rVmove_non_tensor_nodes_on_boundary..can_move_node_and_dependencies..dfssw& KK %#33/ = ;.!]2!!,/!-\:H!8J"r#)set) rnrrrrrrrrs ``@@@@r"can_move_node_and_dependenciesImove_non_tensor_nodes_on_boundary..can_move_node_and_dependenciesxs6 %  > D &&r#rz:Only node that has children in other subgraph can be movedr zCannot move non-tensor node z: on boundary because it has children in multiple subgraphsz"In order move the non-tensor node z on boundary, moved node z from accgpu_z to zParent z) should have children in another subgraphN)r{rqrr:rAr?r=boolrvris_accr rLpoprrpprintr remove) subgraphsrrjrnrrr subgraph_idxqueue processedrrtarget_subgraph_candidatesrchild_subgraphtarget_subgraphrr node_to_moveparentsparentrrs @@r"rrAs},24 + NND%& T "#, EHHMM d588==6I  588== T%((--5H   hhmm  36     1'hhmm1',/1'>A1' tS'' (1'1'h#,I"6  %'(+ NND$T**.dAA T"#e 99QB" -.!32<3D3D2EEA8<<>O'EO' #Hm$1LNN)),7.44;;LI5D$\2<\=N=Nccdeqdrs'0'A'H'HeeTTUVeUfh%2/|<%F1&99")3"&66,V4 D >flSS%fkk]2[\S V,&ue%#7r#)rHN)"ra dataclassesrrtypingrrtorch.fxrtorch.fx._compatibilityrtorch.fx.graphrtorch.fx.passes.utilsr r tools_commonr r r__all__rrrr:rBr?r>rr`rvr@rrr4r#r"rsX ("1"GLL e, - e,N-Ne, .. -.0e, %161E1E t t s)tt t --. t  588  uxx';';T#s(^'K!L LM t-tne,A--A-r#