hPSSKJrJrJr SSKrSSKJr SSKJr SSK J r J r SSK J r Jr SSKJr SSKJrJrJrJrJrJr SS KJrJr S rS r"S S \ 5r"SS\ 5r\r"SS\5r "SS\5r!g))AnyOptionalUnionN)spaces)nn) SquashedDiagGaussianDistributionStateDependentNoiseDistribution) BasePolicyContinuousCritic)get_action_dim)BaseFeaturesExtractorCombinedExtractorFlattenExtractor NatureCNN create_mlpget_actor_critic_arch) PyTorchObsScheduleic\^\rSrSr%Sr\R \S'\RSSSSSS4S\RS\R S \ \ S \RS \ S \\RS \S\S\S\S\S\4U4SjjjrS\\\44U4SjjrS\R.4SjrS"S\ SS4SjjrS\S\\R.\R.\\\R.444SjrS#S\S\S\R.4SjjrS\S\\R.\R.44SjrS#S\S\S\R.4S jjrS!r U=r!$)$Actora# Actor network (policy) for SAC. :param observation_space: Observation space :param action_space: Action space :param net_arch: Network architecture :param features_extractor: Network to extract features (a CNN when using images, a nn.Flatten() layer otherwise) :param features_dim: Number of features :param activation_fn: Activation function :param use_sde: Whether to use State Dependent Exploration or not :param log_std_init: Initial value for the log standard deviation :param full_std: Whether to use (n_features x n_actions) parameters for the std instead of only (n_features,) when using gSDE. :param use_expln: Use ``expln()`` function instead of ``exp()`` when using gSDE to ensure a positive standard deviation (cf paper). It allows to keep variance above zero and prevent it from growing too fast. In practice, ``exp()`` is usually enough. :param clip_mean: Clip the mean output when using gSDE to avoid numerical instability. :param normalize_images: Whether to normalize images or not, dividing by 255.0 (True by default) action_spaceFT@observation_spacenet_archfeatures_extractor features_dim activation_fnuse_sde log_std_initfull_std use_expln clip_meannormalize_imagesc >[TU]UUUU SS9 XplSUlX0lXPlX`lXlXlXl Xl [UR5n [USX65n[R"U6Ul[#U5S:aUSOUnUR(az[%XU SSS9UlUR&R)XUS9uUlUlU S:a;[R"UR*[R."U *U S95Ulgg[1U 5Ul[R2"X5Ul[R2"X5Ulg) NT)rr& squash_outputr)r#r$learn_featuresr() latent_dimlatent_sde_dimr"g)min_valmax_val)super__init__r!sde_features_extractorrrr r"r$r#r%r rrr Sequential latent_pilenr action_distproba_distribution_netmulog_stdHardtanhrLinear)selfrrrrrr r!r"r#r$r%r& action_dim latent_pi_netlast_layer_dim __class__s X/home/james-whalen/.local/lib/python3.13/site-packages/stable_baselines3/sac/policies.pyr0Actor.__init__2sM   1-   &*# (*(" "#D$5$56 "<XM  6),X):"  <<>SWgk D %)$4$4$K$K)Wc%L% !DGT\ 3--iZYb1cd @ KD ii;DG99^@DLreturnc>[TU]5nUR[URUR UR URURURURURURS9 5 U$)N) rrr r!r"r#r$rr%) r/_get_constructor_parametersupdatedictrrr r!r"r#r$rr%r;datar?s r@rE!Actor._get_constructor_parametersisrw24 !.."00 !....#'#:#:..   rBcSn[UR[5(dU5eURRUR5$)a Retrieve the standard deviation of the action distribution. Only useful when using gSDE. It corresponds to ``th.exp(log_std)`` in the normal case, but is slightly different when using ``expln`` function (cf StateDependentNoiseDistribution doc). :return: z+get_std() is only available when using gSDE) isinstancer5r get_stdr8)r;msgs r@rM Actor.get_std{sC<$**,KLLQcQL'' 55rB batch_sizeNcSn[UR[5(dU5eURRURUS9 g)U Sample new weights for the exploration matrix, when using gSDE. :param batch_size: z/reset_noise() is only available when using gSDErPN)rLr5r sample_weightsr8)r;rPrNs r@ reset_noiseActor.reset_noisesF @$**,KLLQcQL '' 'LrBobsc2URXR5nURU5nURU5nUR(aX@R [ US94$UR U5n[R"U[[5nXE04$)z Get the parameters for the action distribution. :param obs: :return: Mean, standard deviation and optional keyword arguments. ) latent_sde) extract_featuresrr3r7r!r8rGthclamp LOG_STD_MIN LOG_STD_MAX)r;rWfeaturesr3 mean_actionsr8s r@get_action_dist_paramsActor.get_action_dist_paramss|((.E.EFNN8, wwy) <<ty/II I,,y)((7K=b((rB deterministicchURU5up4nURR"X44SU0UD6$)Nrc)rar5actions_from_params)r;rWrcr`r8kwargss r@forward Actor.forwards:(,(C(CC(H% v33LqYfqjpqqrBcdURU5up#nURR"X#40UD6$N)rar5log_prob_from_params)r;rWr`r8rfs r@action_log_probActor.action_log_probs3(,(C(CC(H% v44\UfUUrB observationcU"X5$rjr;rnrcs r@_predictActor._predicts K//rB) r5r r%rr#r3r8r"r7rr1r$r!F)"__name__ __module__ __qualname____firstlineno____doc__rBox__annotations__rReLUSpacelistintModuletypeboolfloatr0rGstrrrEr[TensorrMrUrtuplerargrlrr__static_attributes__ __classcell__r?s@r@rrs,***, !%5A!<<5Ajj5As) 5A II 5A  5ABII5A5A5A5A5A5A5A5AnT#s(^$ 6 6McM$M)*)ryy"))UYZ]_a_h_hZhUi?i9j)(r:rdrryyr V:V% 2998L2MV 0J0t0PRPYPY00rBrc!^\rSrSr%Sr\\S'\\S'\\S'S\RSSSS \ SS \ RRSS S4 S \RS \R S\S\\\\\\\\44S\\R2S\S\S\S\S\\S\\\\4S\S\\ RR<S\\\\4S\S\4 U4SjjjrS\SS4Sjr S\\\44U4Sjjr!S-S \SS4S!jjr"S.S"\\S\4S#jjr#S.S"\\S\4S$jjr$S/S%\%S&\S\ RL4S'jjr'S/S(\%S&\S\ RL4S)jjr(S*\SS4S+jr)S,r*U=r+$)0 SACPolicya Policy class (with both actor and critic) for SAC. :param observation_space: Observation space :param action_space: Action space :param lr_schedule: Learning rate schedule (could be constant) :param net_arch: The specification of the policy and value networks. :param activation_fn: Activation function :param use_sde: Whether to use State Dependent Exploration or not :param log_std_init: Initial value for the log standard deviation :param use_expln: Use ``expln()`` function instead of ``exp()`` when using gSDE to ensure a positive standard deviation (cf paper). It allows to keep variance above zero and prevent it from growing too fast. In practice, ``exp()`` is usually enough. :param clip_mean: Clip the mean output when using gSDE to avoid numerical instability. :param features_extractor_class: Features extractor to use. :param features_extractor_kwargs: Keyword arguments to pass to the features extractor. :param normalize_images: Whether to normalize images or not, dividing by 255.0 (True by default) :param optimizer_class: The optimizer to use, ``th.optim.Adam`` by default :param optimizer_kwargs: Additional keyword arguments, excluding the learning rate, to pass to the optimizer :param n_critics: Number of critic networks to create. :param share_features_extractor: Whether to share or not the features extractor between the actor and the critic (this saves computation time) actorcritic critic_targetNFrrTrrr lr_schedulerr r!r"r$r%features_extractor_classfeatures_extractor_kwargsr&optimizer_classoptimizer_kwargs n_criticsshare_features_extractorc >[TU]UUU U U USU S9 UcSS/n[U5unnX@lXPlUR UR UURU S.UlURR5Ul UUUU S.nURRU5 URR5Ul URRUUUS.5 UUl URU5 g)NT)rrr(r&)rrrr r&)r!r"r$r%)rrr)r/r0rrr rrnet_argscopy actor_kwargsrF critic_kwargsr_build)r;rrrrr r!r"r$r%rrr&rrrr actor_arch critic_arch sde_kwargsr?s r@r0SACPolicy.__init__s&    $ %+--    SzH"7"A K *!%!7!7 --"!// 0   !MM..0(""    ,!]]//1 !!&',D  )A% K rBrCcLUR5UlUR"URR54SU"S50URD6URlUR (a_URURRS9Ul URR5VVs/sHup#SU;dM UPM nnnO7URSS9Ul [URR55nURSS9Ul URRURR55 UR"U4SU"S50URD6URlURRS5 gs snnf)Nlrru)rrF) make_actorrr parametersr optimizerr make_criticrrnamed_parametersrrload_state_dict state_dictset_training_mode)r;rnameparamcritic_parameterss r@rSACPolicy._buildsk__& #33 JJ ! ! # 1~ ##    ( (**djj>[>[*\DK;?++:V:V:X }:X;4\px|\|:X  } **d*CDK $T[[%;%;%= > "---F **4;;+A+A+CD $ 4 4 ! 1~! ##!   ,,U3%!~s 8 F F c~>[TU]5nUR[URUR SUR SUR SUR SUR SURSURURURURURS9 5 U$)Nr r!r"r$r%r) rr r!r"r$r%rrrrrr) r/rErFrGrrrr_dummy_schedulerrrrrHs r@rE%SACPolicy._get_constructor_parameters8sw24 "mmO<)))4!..~>++K8++K8,,[9 00 $ 4 4!%!6!6)-)F)F*.*H*H   rBrPc6URRUS9 g)rRrSN)rrU)r;rPs r@rUSACPolicy.reset_noiseMs *5rBrcURURU5n[S0UD6RUR5$Nrp)_update_features_extractorrrtodevice)r;rrs r@rSACPolicy.make_actorUs866t7H7HJ\] $|$'' 44rBcURURU5n[S0UD6RUR5$r)rrr rr)r;rrs r@rSACPolicy.make_criticYs8778J8JL^_ 0-033DKK@@rBrWrcc URXS9$)N)rc)rr)r;rWrcs r@rgSACPolicy.forward]s}}S}>>rBrnc$URX5$rj)rrqs r@rrSACPolicy._predict`szz+55rBmodec|URRU5 URRU5 Xlg)z Put the policy in either training or evaluation mode. This affects certain modules, such as batch normalisation and dropout. :param mode: if true, set to training mode, else set to evaluation mode N)rrrtraining)r;rs r@rSACPolicy.set_training_modecs- $$T* %%d+ rB) r rrrrrrrrrrtrjrv),rwrxryrzr{rr}r rr~rr[optimAdamrrr|rrrrrrGrrrrrr r Optimizerr0rrErUrrrrrgrrrrrrs@r@rrs58 L ##FJ)+ @P>B!%46HHMM59).#@!!<<@!jj@! @! 5cDd3i,@!@AB @! BII @!@!@!@!@!#''<"=@!$,DcN#;@!@!bhh001@!#4S>2@! !@!"#'#@!@!D4(4t4@T#s(^*6c6$65X6K-L5X]5Ah7L.MAYiA?:?d?ryy?6J6t6PRPYPY6 d t  rBrc!^\rSrSrSrS\R SSSS\SS\RRSSS4 S \ RS \ RS \S \\\\\\\\44S \\R,S\S\S\S\S\\S\\\\4S\S\\RR6S\\\\4S\S\4 U4SjjjrSrU=r$) CnnPolicyis: Policy class (with both actor and critic) for SAC. :param observation_space: Observation space :param action_space: Action space :param lr_schedule: Learning rate schedule (could be constant) :param net_arch: The specification of the policy and value networks. :param activation_fn: Activation function :param use_sde: Whether to use State Dependent Exploration or not :param log_std_init: Initial value for the log standard deviation :param use_expln: Use ``expln()`` function instead of ``exp()`` when using gSDE to ensure a positive standard deviation (cf paper). It allows to keep variance above zero and prevent it from growing too fast. In practice, ``exp()`` is usually enough. :param clip_mean: Clip the mean output when using gSDE to avoid numerical instability. :param features_extractor_class: Features extractor to use. :param normalize_images: Whether to normalize images or not, dividing by 255.0 (True by default) :param optimizer_class: The optimizer to use, ``th.optim.Adam`` by default :param optimizer_kwargs: Additional keyword arguments, excluding the learning rate, to pass to the optimizer :param n_critics: Number of critic networks to create. :param share_features_extractor: Whether to share or not the features extractor between the actor and the critic (this saves computation time) NFrrTrrrrrr r!r"r$r%rrr&rrrrcB>[TU]UUUUUUUUU U U U U UUU5 grjr/r0r;rrrrr r!r"r$r%rrr&rrrrr?s r@r0CnnPolicy.__init__B&           $ %     $! rBrp)rwrxryrzr{rr~rr[rrrrr|rrrrrrGrrrrrr rrr0rrrs@r@rrssF>FJ)+ @I>B!%46HHMM59).#$ !<<$ jj$  $ 5cDd3i,@!@AB $ BII $ $ $ $ $ #''<"=$ $,DcN#;$ $ bhh001$ #4S>2$ !$ "#'#$ $ rBrc!^\rSrSrSrS\R SSSS\SS\RRSSS4 S \ RS \ RS \S \\\\\\\\44S \\R,S\S\S\S\S\\S\\\\4S\S\\RR6S\\\\4S\S\4 U4SjjjrSrU=r$)MultiInputPolicyirNFrrTrrrrrr r!r"r$r%rrr&rrrrcB>[TU]UUUUUUUUU U U U U UUU5 grjrrs r@r0MultiInputPolicy.__init__rrBrp)rwrxryrzr{rr~rr[rrrrr|rrrrrrGrrrrrr rrr0rrrs@r@rrsF>FJ)+ @Q>B!%46HHMM59).#$ !<<$ jj$  $ 5cDd3i,@!@AB $ BII $ $ $ $ $ #''<"=$ $,DcN#;$ $ bhh001$ #4S>2$ !$ "#'#$ $ rBr)"typingrrrtorchr[ gymnasiumrr&stable_baselines3.common.distributionsrr !stable_baselines3.common.policiesr r &stable_baselines3.common.preprocessingr %stable_baselines3.common.torch_layersr rrrrr%stable_baselines3.common.type_aliasesrrr^r]rr MlpPolicyrrrprBr@rsw''tJAG  Y0JY0xx xv  ? ? D? y? rB