h SSKJrJrJrJr SSKrSSKJr SSK J r SSK J r J r Jr SSKJr SSKJr \"SS S 9r"S S \5rg) )AnyOptionalTypeVarUnionN) ReplayBuffer) ActionNoise)GymEnv MaybeCallbackSchedule) TD3Policy)TD3SelfDDPGDDPG)boundc+^\rSrSrSrS#S\\\\4S\\ \4S\\ \ 4S\ S\ S\ S \ S \ S \\ \ \ \44S \ S \\S\\\S\\\\4S\S\ S\\S\\\\4S\ S\\ S\\R*\4S\4*U4SjjjrS$S\S\ S\S\ S\S\S\S \4U4S!jjjrS"rU=r$)%ra& Deep Deterministic Policy Gradient (DDPG). Deterministic Policy Gradient: http://proceedings.mlr.press/v32/silver14.pdf DDPG Paper: https://arxiv.org/abs/1509.02971 Introduction to DDPG: https://spinningup.openai.com/en/latest/algorithms/ddpg.html Note: we treat DDPG as a special case of its successor TD3. :param policy: The policy model to use (MlpPolicy, CnnPolicy, ...) :param env: The environment to learn from (if registered in Gym, can be str) :param learning_rate: learning rate for adam optimizer, the same learning rate will be used for all networks (Q-Values, Actor and Value function) it can be a function of the current progress remaining (from 1 to 0) :param buffer_size: size of the replay buffer :param learning_starts: how many steps of the model to collect transitions for before learning starts :param batch_size: Minibatch size for each gradient update :param tau: the soft update coefficient ("Polyak update", between 0 and 1) :param gamma: the discount factor :param train_freq: Update the model every ``train_freq`` steps. Alternatively pass a tuple of frequency and unit like ``(5, "step")`` or ``(2, "episode")``. :param gradient_steps: How many gradient steps to do after each rollout (see ``train_freq``) Set to ``-1`` means to do as many gradient steps as steps done in the environment during the rollout. :param action_noise: the action noise type (None by default), this can help for hard exploration problem. Cf common.noise for the different action noise type. :param replay_buffer_class: Replay buffer class to use (for instance ``HerReplayBuffer``). If ``None``, it will be automatically selected. :param replay_buffer_kwargs: Keyword arguments to pass to the replay buffer on creation. :param optimize_memory_usage: Enable a memory efficient variant of the replay buffer at a cost of more complexity. See https://github.com/DLR-RM/stable-baselines3/issues/37#issuecomment-637501195 :param n_steps: When n_step > 1, uses n-step return (with the NStepReplayBuffer) when updating the Q-value network. :param policy_kwargs: additional arguments to be passed to the policy on creation. See :ref:`ddpg_policies` :param verbose: Verbosity level: 0 for no output, 1 for info messages (such as device or wrappers used), 2 for debug messages :param seed: Seed for the pseudo random generators :param device: Device (cpu, cuda, ...) on which the code should be run. Setting it to auto, the code will be run on the GPU if possible. :param _init_setup_model: Whether or not to build the network at the creation of the instance policyenv learning_rate buffer_sizelearning_starts batch_sizetaugamma train_freqgradient_steps action_noisereplay_buffer_classreplay_buffer_kwargsoptimize_memory_usagen_stepstensorboard_log policy_kwargsverboseseeddevice_init_setup_modelc>[TU]UUUUUUUUU U U U U UUUUUUUSSSSS9 SUR;aSURS'U(aUR5 gg)Ngg?F)rrrrrrrrrrrrrr r!r#r"r$r&r% policy_delaytarget_noise_cliptarget_policy_noiser' n_critics)super__init__r# _setup_model)selfrrrrrrrrrrrrrr r!r"r#r$r%r&r' __class__s U/home/james-whalen/.local/lib/python3.13/site-packages/stable_baselines3/ddpg/ddpg.pyr/ DDPG.__init__9s0 '#+!!)% 3!5"7'+! ##5  < d00 0./D  { +      r1total_timestepscallback log_interval tb_log_namereset_num_timesteps progress_barreturnc (>[TU]UUUUUUS9$)N)r6r7r8r9r:r;)r.learn)r1r6r7r8r9r:r;r2s r3r> DDPG.learnus-w}+%# 3%   r5)gMbP?i@Bdg{Gzt?gGz?r)r)NNNFr)NNrNautoT)NrTF)__name__ __module__ __qualname____firstlineno____doc__rstrtyper r floatr inttuplerrrdictrboolthr&r/rr r>__static_attributes__ __classcell__)r2s@r3rrs(\15$"23.2<@9=&+)-26"(."&-: c4 ?*+: 63; : UH_- :  :  : : : : #uS#X./: : {+: &d<&89: 'tCH~6:  $: !: ""##: $ S#X/%: &': (sm): *biin%+: , -: : ~#'!$("         "      r5)typingrrrrtorchrQ stable_baselines3.common.buffersrstable_baselines3.common.noiser%stable_baselines3.common.type_aliasesr r r stable_baselines3.td3.policiesr stable_baselines3.td3.td3r rrr@r5r3r[s:0096QQ4) :V ,w 3w r5