hkSSKrSSKrSSKJrJr SSKJrJrJrJ r J r SSK r SSK rSSKJr SSKJr \R&"S\S9 SSKJr SS KJr SS KJrJrJr \(aSS KJr "S S \5r"SS\5r"SS\5r "SS\5r!"SS\5r""SS\5r#"SS\5r$"SS\5r%"SS\%5r&"SS\5r'"S S!\5r("S"S#\5r)g!\a SrNf=f)$N)ABCabstractmethod) TYPE_CHECKINGAnyCallableOptionalUnion)Logger)TqdmExperimentalWarningignore)category)tqdm)evaluate_policy) DummyVecEnvVecEnvsync_envs_normalization) base_classcV^\rSrSr%SrS\S'SS\4U4Sjjjr\S\ 4Sj5r \S\ 4S j5r SS jr SS jrS \\\4S\\\4SS 4SjrSSjrSSjrSSjr\S\4Sj5rS\4SjrSSjrSSjrSSjrSSjrS \\\4SS 4SjrS \\\4SS 4SjrSr U=r!$) BaseCallbackzw Base class for callback. :param verbose: Verbosity level: 0 for no output, 1 for info messages, 2 for debug messages base_class.BaseAlgorithmmodelverbosect>[TU]5 SUlSUlXl0Ul0UlSUlgNr)super__init__n_calls num_timestepsrlocalsglobalsparent)selfr __class__s \/home/james-whalen/.local/lib/python3.13/site-packages/stable_baselines3/common/callbacks.pyrBaseCallback.__init__)s9   &( ')  returncNURR5nUcS5eU$)Nzc`model.get_env()` returned None, you must initialize the model with an environment to use callbacks)rget_env)r# training_envs r%r+BaseCallback.training_env6s2zz))+  $ q p q $r'c.URR$N)rloggerr#s r%r/BaseCallback.logger>szz   r'Nc0XlUR5 g)zl Initialize the callback by saving references to the RL model and the training environment for convenience. N)r_init_callback)r#rs r% init_callbackBaseCallback.init_callbackCs   r'cgr.r0s r%r3BaseCallback._init_callbackK r'locals_globals_crXlX lURRUlUR 5 gr.)r r!rr_on_training_start)r#r:r;s r%on_training_startBaseCallback.on_training_startNs*  !ZZ55 !r'cgr.r7r0s r%r=BaseCallback._on_training_startVr9r'c$UR5 gr.)_on_rollout_startr0s r%on_rollout_startBaseCallback.on_rollout_startYs  r'cgr.r7r0s r%rCBaseCallback._on_rollout_start\r9r'cg)zD :return: If the callback returns False, training is aborted early. Tr7r0s r%_on_stepBaseCallback._on_step_s r'cU=RS- slURRUlUR5$)z This method will be called by the model after each call to ``env.step()``. For child callback (of an ``EventCallback``), this will be called when the event is triggered. :return: If the callback returns False, training is aborted early. )rrrrIr0s r%on_stepBaseCallback.on_stepfs0  !ZZ55}}r'c$UR5 gr.)_on_training_endr0s r%on_training_endBaseCallback.on_training_endts r'cgr.r7r0s r%rPBaseCallback._on_training_endwr9r'c$UR5 gr.)_on_rollout_endr0s r%on_rollout_endBaseCallback.on_rollout_endzs r'cgr.r7r0s r%rVBaseCallback._on_rollout_end}r9r'c\URRU5 URU5 gzn Update the references to the local variables. :param locals_: the local variables during rollout collection N)r updateupdate_child_localsr#r:s r% update_localsBaseCallback.update_localss$ 7#   )r'cg)z Update the references to the local variables on sub callbacks. :param locals_: the local variables during rollout collection Nr7r_s r%r^ BaseCallback.update_child_localss r')r!r rrrr"rrrrr(Nr(N)"__name__ __module__ __qualname____firstlineno____doc____annotations__intrpropertyrr+r r/r4r3dictstrrr>r=rDrCrboolrIrMrQrPrWrVr`r^__static_attributes__ __classcell__r$s@r%rrs &%   f!!! "c3h"4S>"VZ" ! $     *T#s(^** 4S> d  r'rc^\rSrSrSrSS\\S\4U4SjjjrSU4Sjjr SS jr S\ 4S jr S\ 4S jr S \\\4SS4S jrSrU=r$) EventCallbackz Base class for triggering callback on event. :param callback: Callback that will be called when an event is triggered. :param verbose: Verbosity level: 0 for no output, 1 for info messages, 2 for debug messages Ncallbackrcr>[TU]US9 XlUb URceXRlggNr)rrrxr"r#rxrr$s r%rEventCallback.__init__s= )  ==, ,,#'MM  r'r(c>[TU]U5 URb&URRUR5 ggr.)rr4rxr)r#rr$s r%r4EventCallback.init_callbacks5 e$ == $ MM ' ' 3 %r'cURb1URRURUR5 ggr.)rxr>r r!r0s r%r= EventCallback._on_training_starts- == $ MM + +DKK F %r'cRURbURR5$gNT)rxrMr0s r% _on_eventEventCallback._on_events" == $==((* *r'cgrr7r0s r%rIEventCallback._on_stepsr'r:cVURbURRU5 ggr\rxr`r_s r%r^!EventCallback.update_child_localss% == $ MM ' ' 0 %r'rxrrerf)rgrhrirjrkrrrmrr4r=rqrrIrorprr^rrrsrts@r%rvrvsh(,!7(((4 G4 $14S>1d11r'rvc^\rSrSrSrS\\4U4SjjrSSjrSSjr SS jr S\ 4S jr SS jr SS jrS \\\4SS4SjrSrU=r$) CallbackListzl Class for chaining callbacks. :param callbacks: A list of callbacks that will be called sequentially. callbacksc\>[TU]5 [U[5(deXlgr.)rr isinstancelistr)r#rr$s r%rCallbackList.__init__s& )T****"r'r(NcURH/nURUR5 URUlM1 gr.)rr4rr"r#rxs r%r3CallbackList._init_callbacks/H  " "4:: .#kkHO 'r'cvURH)nURURUR5 M+ gr.)rr>r r!rs r%r=CallbackList._on_training_starts(H  & &t{{DLL A'r'cJURHnUR5 M gr.)rrDrs r%rCCallbackList._on_rollout_startsH  % % ''r'cbSnURHnUR5=(a UnM U$r)rrM)r#continue_trainingrxs r%rICallbackList._on_steps3 H ( 0 0 2 H7H '! r'cJURHnUR5 M gr.)rrWrs r%rVCallbackList._on_rollout_endsH  # # %'r'cJURHnUR5 M gr.)rrQrs r%rPCallbackList._on_training_endsH  $ $ &'r'r:cLURHnURU5 M gr\)rr`)r#r:rxs r%r^ CallbackList.update_child_localss H  " "7 +'r')rrf)rgrhrirjrkrrrr3r=rCrqrIrVrProrprr^rrrsrts@r%rrs_#$|"4# *B(!$!&',4S>,d,,r'rc ^\rSrSrSrSS\S\S\S\S\S\4 U4S jjjrSS jr SS \S \S \4Sjjr S \4Sjr Sr U=r $)CheckpointCallbacka Callback for saving a model every ``save_freq`` calls to ``env.step()``. By default, it only saves model checkpoints, you need to pass ``save_replay_buffer=True``, and ``save_vecnormalize=True`` to also save replay buffer checkpoints and normalization statistics checkpoints. .. warning:: When using multiple environments, each call to ``env.step()`` will effectively correspond to ``n_envs`` steps. To account for that, you can use ``save_freq = max(save_freq // n_envs, 1)`` :param save_freq: Save checkpoints every ``save_freq`` call of the callback. :param save_path: Path to the folder where the model will be saved. :param name_prefix: Common prefix to the saved models :param save_replay_buffer: Save the model replay buffer :param save_vecnormalize: Save the ``VecNormalize`` statistics :param verbose: Verbosity level: 0 for no output, 2 for indicating when saving model checkpoint save_freq save_path name_prefixsave_replay_buffersave_vecnormalizerc`>[TU]U5 XlX lX0lX@lXPlgr.)rrrrrrr)r#rrrrrrr$s r%rCheckpointCallback.__init__ s. !""&"4!2r'r(c^URb [R"URSS9 gg)NTexist_ok)rosmakedirsr0s r%r3!CheckpointCallback._init_callbacks# >> % KK 6 &r'checkpoint_type extensionc [RRURURSUUR SU35$)a Helper to get checkpoint path for each type of checkpoint. :param checkpoint_type: empty for the model, "replay_buffer_" or "vecnormalize_" for the other checkpoints. :param extension: Checkpoint file extension (zip for model, pkl for others) :return: Path to the checkpoint _z_steps.)rpathjoinrrr)r#rrs r%_checkpoint_path#CheckpointCallback._checkpoint_path sEww||DNNt/?/?.@/ARSWSeSeRffmnwmx,yzzr'cURUR-S:XGaWURSS9nURR U5 UR S:a[ SU35 UR(a{[URS5(a`URRbIURSSS9nURRU5 UR S :a[ S U35 UR(arURR5bWURS SS9nURR5R U5 UR S:a[ S U35 g )Nrzip)rzSaving model checkpoint to replay_bufferreplay_buffer_pklrLz)Saving model replay buffer checkpoint to vecnormalize_zSaving model VecNormalize to T) rrrrsaverprintrhasattrrrget_vec_normalize_env)r# model_pathreplay_buffer_pathvec_normalize_paths r%rICheckpointCallback._on_step+s9 <<$.. (A -...?J JJOOJ '||q 3J<@A&&74::+O+OTXT^T^TlTlTx%)%:%:;KW\%:%]" --.@A<[TU]U5 Xlgr.)rrrxr|s r%rConvertCallback.__init__Ks ! r'r(cjURb&URURUR5$gr)rxr r!r0s r%rIConvertCallback._on_stepOs( == $==dll; ;r'rrd)rgrhrirjrkrrrorprrqrmrrIrrrsrts@r%rrCsW!(DcNDcN3SUY3Y*Z![!fi!!$r'rc^\rSrSrSrSS\\R\4S\ \ S\ \ S\ S\ S \ \ S \ \ S \ S \ S \ S\ 4U4SjjjrSSjrS\\ \4S\\ \4SS4SjrS\ 4SjrS\\ \4SS4SjrSrU=r$) EvalCallbackiUa  Callback for evaluating an agent. .. warning:: When using multiple environments, each call to ``env.step()`` will effectively correspond to ``n_envs`` steps. To account for that, you can use ``eval_freq = max(eval_freq // n_envs, 1)`` :param eval_env: The environment used for initialization :param callback_on_new_best: Callback to trigger when there is a new best model according to the ``mean_reward`` :param callback_after_eval: Callback to trigger after every evaluation :param n_eval_episodes: The number of episodes to test the agent :param eval_freq: Evaluate the agent every ``eval_freq`` call of the callback. :param log_path: Path to a folder where the evaluations (``evaluations.npz``) will be saved. It will be updated at each evaluation. :param best_model_save_path: Path to a folder where the best model according to performance on the eval env will be saved. :param deterministic: Whether the evaluation should use a stochastic or deterministic actions. :param render: Whether to render or not the environment during evaluation :param verbose: Verbosity level: 0 for no output, 1 for indicating information about evaluation results :param warn: Passed to ``evaluate_policy`` (warns if ``eval_env`` has not been wrapped with a Monitor wrapper) Neval_envcallback_on_new_bestcallback_after_evaln_eval_episodes eval_freqlog_pathbest_model_save_path deterministicrenderrwarnc >^[T U]X:S9 X lURbXRlX@lXPl[ R*Ul[ R*Ul Xl Xl Xl [T[5(d[U4Sj/5mTUlXplUb [$R&R)US5nX`l/Ul/Ul/Ul/Ul/Ulg)Nr{c>T$r.r7)rsr%'EvalCallback.__init__..sHr' evaluations)rrrr"rrnpinfbest_mean_rewardlast_mean_rewardrrrrrrrrrrrrevaluations_resultsevaluations_timestepsevaluations_length_is_success_bufferevaluations_successes) r#rrrrrrrrrrrr$s ` r%rEvalCallback.__init__qs ,>$8!  $ $ 0/3 % % ,."!#!#*  (F++"$4#56H  $8!  ww||Hm>A $,,"?1"Dzz//1=+D,=,=t}}M')D #/>  $ 4 4{{"00'+YY33 0 ,O}}(!/48888!/48888**11$2D2DE((//@''..?t../!3..55d6M6MN!D,F,FGFMM"88 44#66   ')ggo&>@W,.GGO,Dbff_F]M$)+$6D !||q +D,>,>+??TU`adTeejkuvyjz{|(('>? <<1$N3+=c*B!DE ""#6 E KK  5t7I7IS`  a KK  T// 0222<<1$12,,8JJOOBGGLL1J1JL$YZ(-k(:%,,8(,(A(A(I(I(K%}}($5$J$..:J!  [&(- s Q?? R RRc^UR(aURRU5 ggr\rr_s r%r^ EvalCallback.update_child_localss" == MM ' ' 0 r')rrrrrrrrrrrrrrrr) NNi'NNTFrLTrf)rgrhrirjrkr gymEnvrrrrmrprqrr3rorrrIr^rrrsrts@r%rrUs#<8<6: "&.2",:(,:'|4,:&l3 ,:  ,:  ,:3-,:'sm,:,:,:,:,:,:\ @AT#s(^AtCQTH~AZ^A U!$U!n14S>1d11r'rcV^\rSrSr%Sr\\S'S S\S\4U4Sjjjr S\ 4Sjr S r U=r $) StopTrainingOnRewardThresholdiat Stop the training once a threshold in episodic reward has been reached (i.e. when the model is good enough). It must be used with the ``EvalCallback``. :param reward_threshold: Minimum expected reward per episode to stop training. :param verbose: Verbosity level: 0 for no output, 1 for indicating when training ended because episodic reward threshold reached r"reward_thresholdrc,>[TU]US9 Xlgrz)rrr")r#r"rr$s r%r&StopTrainingOnRewardThreshold.__init__.s ) 0r'r(cURcS5e[URRUR:5nURS:a7U(d0[ SURRSSUR35 U$)NzN``StopTrainingOnMinimumReward`` callback must be used with an ``EvalCallback``rLz*Stopping training because the mean reward rz is above the threshold )r"rqrr"rrr#rs r%rI&StopTrainingOnRewardThreshold._on_step2s{{&x(xx& !=!=@U@U!UV <<1 %6 >[TU]U5 XlSUlgr)rrr*last_time_trigger)r#r*rxr$s r%rEveryNTimesteps.__init__Fs " !"r'r(cURUR- UR:a!URUlUR5$gr)rr,r*rr0s r%rIEveryNTimesteps._on_stepKs=   !7!7 7DLL H%)%7%7D ">># #r')r,r*) rgrhrirjrkrmrrrqrIrrrsrts@r%r)r)=s,##|# $r'r)c^^\rSrSrSrS\4U4SjjrS\\\ 4S\\\ 4S\ 4Sjr S r U=r $) LogEveryNTimestepsiRz` Log data every ``n_steps`` timesteps :param n_steps: Number of timesteps between two trigger. r*cH>[TU]U[UR5S9 g)Nr)rrr _log_data)r#r*r$s r%rLogEveryNTimesteps.__init__Ys ?4>>+JKr'_locals_globalsr(c8URR5 gr)r dump_logs)r#r5r6s r%r3LogEveryNTimesteps._log_data\s r'r7)rgrhrirjrkrmrrorprrqr3rrrsrts@r%r1r1RsE LLc3h4S>dr'r1cT^\rSrSrSrS S\S\4U4SjjjrS SjrS\4Sjr S r U=r $) StopTrainingOnMaxEpisodesiaa Stop the training once a maximum number of episodes are played. For multiple environments presumes that, the desired behavior is that the agent trains on each env for ``max_episodes`` and in total for ``max_episodes * n_envs`` episodes. :param max_episodes: Maximum number of episodes to stop training. :param verbose: Verbosity level: 0 for no output, 1 for indicating information about when training ended by reaching ``max_episodes`` max_episodesrcF>[TU]US9 XlXlSUlgNr{r)rrr<_total_max_episodes n_episodes)r#r<rr$s r%r"StopTrainingOnMaxEpisodes.__init__ms& )(#/ r'r(cTURURR-Ulgr.)r<r+num_envsr?r0s r%r3(StopTrainingOnMaxEpisodes._init_callbackss!#'#4#4t7H7H7Q7Q#Q r'c JSUR;dS5eU=R[R"URS5R 5- slURUR :nUR S:aU(dURURR- nURRS:aSUSS3OSn[SURS URRS 5S URS URS U3 5 U$)NdoneszT`dones` variable is not defined, please check your code next to `callback.on_step()`rLzwith an average of rz episodes per envrz"Stopping training with a total of z steps because the tb_log_namez model reached max_episodes=z, by playing for z episodes ) r r@rsumitemr?rr+rCrrrr<)r#rmean_episodes_per_env mean_ep_strs r%rI"StopTrainingOnMaxEpisodes._on_stepws$++%}'}}% 266$++g"67<<>> OOd.F.FF <<1 %6$(OOd6G6G6P6P$P !VZVgVgVpVpstVt%&;C%@@QRz|  4T5G5G4HH[;;??=122NtO`O`Nab""&//!2*-!  ! r')r?r<r@rdrf) rgrhrirjrkrmrr3rqrIrrrsrts@r%r;r;as7 S3 R!$!!r'r;cZ^\rSrSr%Sr\\S'S S\S\S\4U4SjjjrS\ 4S jr S r U=r $) StopTrainingOnNoModelImprovementiaM Stop the training early if there is no new best model (new best mean reward) after more than N consecutive evaluations. It is possible to define a minimum number of evaluations before start to count evaluations without improvement. It must be used with the ``EvalCallback``. :param max_no_improvement_evals: Maximum number of consecutive evaluations without a new best model. :param min_evals: Number of evaluations before start to count evaluations without improvements. :param verbose: Verbosity level: 0 for no output, 1 for indicating when training ended because no new best model r"max_no_improvement_evals min_evalsrcr>[TU]US9 XlX l[R *UlSUlgr>)rrrOrPrrlast_best_mean_rewardno_improvement_evals)r#rOrPrr$s r%r)StopTrainingOnNoModelImprovement.__init__s4 )(@%"&(ffW"$%!r'r(cURcS5eSnURUR:a]URRUR:aSUlO1U=R S- slUR UR :aSnURRUlURS:a!U(d[SUR SS35 U$) NzS``StopTrainingOnNoModelImprovement`` callback must be used with an ``EvalCallback``TrrLFzBStopping training because there was no new best model in the last dz evaluations) r"rrPrrRrSrOrrr&s r%rI)StopTrainingOnNoModelImprovement._on_steps{{&}(}}&  <<$.. ({{++d.H.HH,-)))Q.),,t/L/LL(-%%)[[%A%A" <<1 %6 TUYUnUnopTqq}~ ! r')rRrOrPrS)rr) rgrhrirjrkrrlrmrrqrIrrrsrts@r%rNrNsA  &&&SV&&!$!!r'rNc^^\rSrSr%Sr\\S'S U4SjjrS SjrS\ 4Sjr S Sjr S r U=r $) ProgressBarCallbackizN Display a progress bar when training SB3 agent using tqdm and rich packages. pbarr(cF>[TU]5 [c [S5eg)NzYou must install tqdm and rich in order to use the progress bar callback. It is included if you install stable-baselines with the extra packages: `pip install stable-baselines3[extra]`)rrr ImportError)r#r$s r%rProgressBarCallback.__init__s*  <9  r'ch[URSURR- S9Ulg)Ntotal_timesteps)total)rr rrrZr0s r%r=&ProgressBarCallback._on_training_starts)t{{+<= @X@XXY r'cbURRURR5 gr)rZr]r+rCr0s r%rIProgressBarCallback._on_steps# **334r'clURR5 URR5 gr.)rZrefreshcloser0s r%rP$ProgressBarCallback._on_training_ends   r')rZrf)rgrhrirjrkrrlrr=rqrIrPrrrsrts@r%rYrYs0 JZ $ r'rY)*rrabcrrtypingrrrrr gymnasiumrnumpyrstable_baselines3.common.loggerr rr filterwarnings tqdm.richr\#stable_baselines3.common.evaluationr stable_baselines3.common.vec_envrrrstable_baselines3.commonrrrvrrrrr!r)r1r;rNrYr7r'r%rrs #@@2 , H/FG@YY3q 3q h)1L)1X3,<3,lLL^l$G1=G1T!L!<m*  )! )!X*!|*!Z,S DsC--C87C8