h SrSSKrSSKJr SSKJr SSKrSSKJ r J r "SS\R5r "SS\R5r S&S jrS'S jrS(Sjr\S:XGa \"S5 \"S5 \"S5 \ "SSS/S S9r\"S\R(R*35 \"\SSSS9 \"S5 \"S5 \"S5 \"SSSS9urr\"\\\SS SS9 \"SS SS9urr\R55 \R6"5 \"\5r\R;S S!9r\\:HR?5RA5RC5S-r"SSS5 \"S"\"S#S$35 \"S%5 gg!,(df  N%=f))zv Transfer Learning - Reuse learned features for new tasks Core idea: Features learned on Task A are useful for Task B N)ListTuplecD^\rSrSrSrS\S\\4U4SjjrSrSr U=r $)FeatureExtractor zS Pre-trainable feature extractor Learn general features that transfer across tasks input_size feature_sizescX>[TU]5 /nUnUHsnUR[R"XE55 UR[R "55 UR[R "U55 UnMu [R"U6UlX@l gN) super__init__appendnnLinearReLU BatchNorm1d Sequentialfeatures feature_dim)selfrr layers prev_size feature_size __class__s [/home/james-whalen/eden-agi-project/real_capabilities/learning/transfer/transfer_learner.pyr FeatureExtractor.__init__s{  )L MM"))I< = MM"'') $ MM"..6 7$I *  v. $c$URU5$r )rrxs rforwardFeatureExtractor.forward s}}Qr)rr) __name__ __module__ __qualname____firstlineno____doc__intrr r!__static_attributes__ __classcell__rs@rrr s) %3 %tCy %  rrct^\rSrSrSrSS/S4S\S\\S\4U4S jjjrS rS r S r S\4S jr Sr Sr U=r$)TransferLearner#zD Transfer learning model with frozen/fine-tunable feature extractor @ rr n_classesc>[TU]5 [X5Ul[R "URR U5Ulgr )r r rfeature_extractorrrr classifier)rrr r2rs rr TransferLearner.__init__(s; !1*!L))D$:$:$F$F RrcFURU5nURU5$r )r4r5)rr rs rr!TransferLearner.forward3s!))!,x((rcTURR5H nSUlM g)z.Freeze feature extractor for transfer learningFNr4 parameters requires_gradrparams rfreeze_featuresTransferLearner.freeze_features7s#++668E"'E 9rcTURR5H nSUlM g)zUnfreeze for fine-tuningTNr:r=s runfreeze_features!TransferLearner.unfreeze_features<s#++668E"&E 9rcd[R"URRU5Ulg)z$Replace classifier head for new taskN)rrr4rr5)rr2s rreplace_classifier"TransferLearner.replace_classifierAs ))D$:$:$F$F Rrc[R"5 URU5sSSS5 $!,(df  g=f)zGet feature representationsN)torchno_gradr4rs rextract_features TransferLearner.extract_featuresEs# ]]_))!,__s1 ?)r5r4)r#r$r%r&r'r(rr r!r?rBrErJr)r*r+s@rr-r-#sj%(9 S SCy S S S)( ' SCS--rr-2c [S5 [S5 [SUSUSU35 [S5 [R"UR5SS9n[ U5GHnS nS nS n[ U5Hn [ XS S 9upUR 5 U"U 5n [RRX5n U R5 UR5 XmR5- nU RS S 9nX~U :HR5R5- nU[U 5- nM US -S-S :XdMXa- nSU-U- n[SUS -SUSUSSUSS3 5 GM [S5 [S5 g)zY Pre-train feature extractor on multiple source tasks This creates transferable features G ======================================================================z.PRE-TRAINING FEATURE EXTRACTOR ON SOURCE TASKSzTasks: z | Samples per task: z | Epochs: F======================================================================gMbP?lrrrL)rdimr1Y@Epoch /z - Loss: z.4f - Acc: .2f%u ✅ Pre-training complete!z9 Feature extractor now has transferable representationsN)printoptimAdamr;rangegenerate_task_data zero_gradr functional cross_entropybackwardstepitemargmaxsumlen)modeln_taskssamples_per_taskepochs optimizerepoch total_loss total_correct total_samplestask_idXyoutputslosspredavg_lossaccuracys rpretrain_on_source_tasksr{Js  - :; GG912B1C;vh WX &M 5++-%8Iv   W~G%gBODA    !AhG==..w:D MMO NN  ))+ %J>>a>(D ai__.335 5M SV #M#&& AI q !+Hm+m;H F57)1VHIhs^8HUX>YZ[ \9< () EFrTc 2[S5 [S5 [SUSU35 [S5 URU5 U(aUR5 [S5 O [S5 [R"UR 5Vs/sHofR (dMUPM snSS 9n[RRRX5n[RRRUS S S 9n [U5Hn S n S n S n U HupUR5 U"U5n[RR!UU5nUR#5 UR%5 U UR'5- n UR)SS9nU UU:HR+5R'5- n U [-U5- n M U S-S-S :XdMSU -U - n[SU S-SUSUSS35 M [S5 gs snf)z/ Transfer to new target task with few examples rOz TRANSFER LEARNING TO TARGET TASKzTarget classes: z | Freeze features: rPz- Features frozen - training classifier onlyz Fine-tuning all parametersg{Gz?rQ T) batch_sizeshufflerrSrTrVrWrXrYrZr[u ✅ Transfer learning complete!N)r\rEr?r]r^r;r<rHutilsdata TensorDataset DataLoaderr_rarrbrcrdrerfrgrhri)rjtarget_Xtarget_yn_classes_targetr?fine_tune_epochsprndatasetloaderrorpcorrecttotalinputstargetsrvrwrxrzs rtransfer_to_target_taskrws - ,. -..B?BS TU &M -.  => -. $$&:&q//&: I kk,,X@G [[   ( (R ( NF'( %OF    !FmG==..w@D MMO NN  $))+ %J>>a>(D ,,.335 5G S\ !E & AI?a g~-H F57)1%5$6hxnAN O')* -.9 ;s H,Hr1c@[RRUS-5 [RRX#5n[RRX5nXT-nUR SS9n[ R "U5[ R"U54$)zGenerate synthetic task datadrS)axis)nprandomseedrandnrgrH FloatTensor LongTensor)rs n_samplesrr2Wrtlogitsrus rr`r`svIINN7S=!  .A  .A UF 1 A   Q !1!1!!4 44r__main__rOzTRANSFER LEARNING TESTrPr/r0)rr r2z Model created - Feature dim: i)rkrlrmzTESTING TRANSFER TO NEW TASKirr)rr2)rr?rrSrTz Test Accuracy: rZr[u+ ✅ Transfer learning implementation ready)rLirM)TrL)rLr1)#r'rHtorch.nnr torch.optimr]numpyrtypingrrModulerr-r{rr`r#r\rjr4rrrtest_Xtest_yevalrIrvrgrxfloatmeanrfrzrrrs  ryy 0%-bii%-N+Gd 7/r 5 z - "# &M r#rb QE +E,C,C,O,O+P QRUBRP - () &M+C3!LHhE8X[_rtu(rQGNFF JJL -~~!~$FN))+002779C?   hs^1 -. 89;. s A E.. E<