h)SSKJrJr SSKrSSKJr SSKJr SSKJ r SSK J r J r J r JrJr SSKJrJr "SS \ 5r"S S \ 5r\r"S S \5r"SS\5rg))AnyOptionalN)spaces)nn) BasePolicy)BaseFeaturesExtractorCombinedExtractorFlattenExtractor NatureCNN create_mlp) PyTorchObsSchedulecN^\rSrSr%Sr\R \S'S\RS4S\RS\R S\ S\ S \ \\ S \\R S \S S4U4S jjjrS\S \R*4SjrSS\S\S \R*4SjjrS \\\44U4SjjrSrU=r$)QNetworkaL Action-Value (Q-Value) network for DQN :param observation_space: Observation space :param action_space: Action space :param net_arch: The specification of the policy and value networks. :param activation_fn: Activation function :param normalize_images: Whether to normalize images or not, dividing by 255.0 (True by default) action_spaceNTobservation_spacefeatures_extractor features_dimnet_arch activation_fnnormalize_imagesreturnc>[T U]UUUUS9 UcSS/nXPlX`lX@l[ UR R5n[URXRUR5n [R"U 6Ul g)N)rr@) super__init__rrrintrnr r Sequentialq_net) selfrrrrrrr action_dimr! __class__s X/home/james-whalen/.local/lib/python3.13/site-packages/stable_baselines3/dqn/policies.pyrQNetwork.__init__ s   1-    BxH *(**,,- 4,,j--I[I[\]]E* obscVURURXR55$)z` Predict the q-values. :param obs: Observation :return: The estimated Q-Value for each action. )r!extract_featuresr)r"r(s r%forwardQNetwork.forward;s$zz$//5L5LMNNr' observation deterministiccRU"U5nURSS9RS5nU$)N)dim)argmaxreshape)r"r-r.q_valuesactions r%_predictQNetwork._predictDs, $Q'//3 r'c >[TU]5nUR[URUR UR URS95 U$)N)rrrr)r_get_constructor_parametersupdatedictrrrrr"datar$s r%r:$QNetwork._get_constructor_parametersJsMw24 !.."00#'#:#:    r')rrrr!T)__name__ __module__ __qualname____firstlineno____doc__rDiscrete__annotations__rReLUSpacerrrlisttypeModuleboolrr thTensorr+r7r<strrr:__static_attributes__ __classcell__r$s@r%rrs //!)-)+!%+!<<+oo+2 +  + 49% +BII++ ++6O:O"))OJtryy T#s(^  r'rc^\rSrSr%Sr\\S'\\S'S\R\ SS\ RRS4S\ RS\ RS \S \\\S \\R*S \\S \\\\4S\S\\ RR6S\\\\4SS4U4SjjjrS \SS4SjrS\4SjrSS\S\S\ R@4Sjjr!SS\S\S\ R@4Sjjr"S\\\44U4Sjjr#S\SS4Sjr$Sr%U=r&$) DQNPolicyXa Policy class with Q-Value Net and target net for DQN :param observation_space: Observation space :param action_space: Action space :param lr_schedule: Learning rate schedule (could be constant) :param net_arch: The specification of the policy and value networks. :param activation_fn: Activation function :param features_extractor_class: Features extractor to use. :param features_extractor_kwargs: Keyword arguments to pass to the features extractor. :param normalize_images: Whether to normalize images or not, dividing by 255.0 (True by default) :param optimizer_class: The optimizer to use, ``th.optim.Adam`` by default :param optimizer_kwargs: Additional keyword arguments, excluding the learning rate, to pass to the optimizer r! q_net_targetNTrr lr_schedulerrfeatures_extractor_classfeatures_extractor_kwargsroptimizer_classoptimizer_kwargsrc >[T U]UUUUU U US9 UcU[:Xa/nOSS/nX@lXPlUR UR URURUS.UlURU5 g)N)r[r\rr)rrrrr) rrr rrrrnet_args_build r"rrrXrrrYrZrr[r\r$s r%rDQNPolicy.__init__os    $ %+--    '948 *"&!7!7 -- !// 0    K r'cxUR5UlUR5UlURRURR 55 URR S5 UR "URR54SU"S50URD6Ul g)z Create the network and the optimizer. Put the target network into evaluation mode. :param lr_schedule: Learning rate schedule lr_schedule(1) is the initial learning rate Flrr0N) make_q_netr!rWload_state_dict state_dictset_training_moder[ parametersr\ optimizer)r"rXs r%r_DQNPolicy._builds__&  OO- ))$***?*?*AB ++E2-- JJ ! ! # 1~ ## r'c~URURSS9n[S0UD6RUR5$)N)r)_update_features_extractorr^rtodevice)r"r^s r%rdDQNPolicy.make_q_nets9224==UY2Z#(#&&t{{33r'r(r.c URXS9$N)r.)r7r"r(r.s r%r+DQNPolicy.forwards}}S}>>r'c4URRXS9$rr)r!r7rss r%r7DQNPolicy._predictszz""3"DDr'c >[TU]5nUR[URSURSUR UR URURURS95 U$)Nrr)rrrXr[r\rYrZ) rr:r;r<r^_dummy_scheduler[r\rYrZr=s r%r:%DQNPolicy._get_constructor_parametersspw24 z2"mmO< 00 $ 4 4!%!6!6)-)F)F*.*H*H   r'modecFURRU5 Xlg)z Put the policy in either training or evaluation mode. This affects certain modules, such as batch normalisation and dropout. :param mode: if true, set to training mode, else set to evaluation mode N)r!rgtraining)r"rzs r%rgDQNPolicy.set_training_modes $$T* r')rrr^rir!rWr|r@)'rArBrCrDrErrGrrHr rNoptimAdamrrIrFrrrJrrKrLrr<rPrrM Optimizerrr_rdr rOr+r7r:rgrQrRrSs@r%rUrUXs& O)-)+@P>B!%46HHMM59(!!<<(!oo(! (! 49% (! BII (!#''<"=(!$,DcN#;(!(!bhh001(!#4S>2(! (!(!T ( t ,4H4 ?:?d?bii?EJEtEryyET#s(^ d t  r'rUcL^\rSrSrSrS\R \SS\RRS4S\ RS\ RS\S\\\S \\R&S \\S \\\\4S \S \\RR2S\\\\4SS4U4SjjjrSrU=r$) CnnPolicya Policy class for DQN when using images as input. :param observation_space: Observation space :param action_space: Action space :param lr_schedule: Learning rate schedule (could be constant) :param net_arch: The specification of the policy and value networks. :param activation_fn: Activation function :param features_extractor_class: Features extractor to use. :param normalize_images: Whether to normalize images or not, dividing by 255.0 (True by default) :param optimizer_class: The optimizer to use, ``th.optim.Adam`` by default :param optimizer_kwargs: Additional keyword arguments, excluding the learning rate, to pass to the optimizer NTrrrXrrrYrZrr[r\rc 6>[T U]UUUUUUUUU U 5 gNrrr`s r%rCnnPolicy.__init__0       $ %    r'rl)rArBrCrDrErrHr rNr~rrrIrFrrrJrrKrLrr<rPrrMrrrQrRrSs@r%rrs,)-)+@I>B!%46HHMM59 !<< oo   49%  BII  #''<"= $,DcN#;  bhh001 #4S>2    r'rcL^\rSrSrSrS\R \SS\RRS4S\ RS\ RS\S\\\S \\R&S \\S \\\\4S \S \\RR2S\\\\4SS4U4SjjjrSrU=r$)MultiInputPolicyia Policy class for DQN when using dict observations as input. :param observation_space: Observation space :param action_space: Action space :param lr_schedule: Learning rate schedule (could be constant) :param net_arch: The specification of the policy and value networks. :param activation_fn: Activation function :param features_extractor_class: Features extractor to use. :param normalize_images: Whether to normalize images or not, dividing by 255.0 (True by default) :param optimizer_class: The optimizer to use, ``th.optim.Adam`` by default :param optimizer_kwargs: Additional keyword arguments, excluding the learning rate, to pass to the optimizer NTrrrXrrrYrZrr[r\rc 6>[T U]UUUUUUUUU U 5 grrr`s r%rMultiInputPolicy.__init__rr'rl)rArBrCrDrErrHr rNr~rrDictrFrrrJrrKrLrr<rPrrMrrrQrRrSs@r%rrs,)-)+@Q>B!%46HHMM59 !;; oo   49%  BII  #''<"= $,DcN#;  bhh001 #4S>2    r'r)typingrrtorchrN gymnasiumrr!stable_baselines3.common.policiesr%stable_baselines3.common.torch_layersrr r r r %stable_baselines3.common.type_aliasesr rrrU MlpPolicyrrrlr'r%rsb 8GCzCL{ {|  * * Z* y* r'