hs SrSSKJr SSKJr SSKJrJrJrJ r J r SSK r SSK r SSK Jr SSKJrJr \(aSSKJrJr \ "S 5r\ "S 5r\ "S 5r"S S \\\45r\ "S5r\ "S5r"SS\\\4\\\\\45r"SS\\\\\45r"SS\\\\\45r"SS\\\\\45rg)zWCore API for Environment, Wrapper, ActionWrapper, RewardWrapper and ObservationWrapper.) annotations)deepcopy) TYPE_CHECKINGAnyGeneric SupportsFloatTypeVarN)spaces)RecordConstructorArgsseeding)EnvSpec WrapperSpecObsTypeActType RenderFramecX\rSrSr%SrS/0rS\S'SrS\S'SrS \S 'S \S 'S \S'Sr S\S'Sr S\S'S%Sjr SSS.S&Sjjr S'Sjr Sr\S(Sj5r\S)Sj5r\S*Sj5r\R&S+Sj5rSrSrS,SjrS-SjrS.S jrS!S".S/S#jjrS$rg)0Enva The main Gymnasium class for implementing Reinforcement Learning Agents environments. The class encapsulates an environment with arbitrary behind-the-scenes dynamics through the :meth:`step` and :meth:`reset` functions. An environment can be partially or fully observed by single agents. For multi-agent environments, see PettingZoo. The main API methods that users of this class need to know are: - :meth:`step` - Updates an environment with actions returning the next agent observation, the reward for taking that actions, if the environment has terminated or truncated due to the latest action and information from the environment about the step, i.e. metrics, debug info. - :meth:`reset` - Resets the environment to an initial state, required before calling step. Returns the first agent observation for an episode and information, i.e. metrics, debug info. - :meth:`render` - Renders the environments to help visualise what the agent see, examples modes are "human", "rgb_array", "ansi" for text. - :meth:`close` - Closes the environment, important when external software is used, i.e. pygame for rendering, databases Environments have additional attributes for users to understand the implementation - :attr:`action_space` - The Space object corresponding to valid actions, all valid actions should be contained within the space. - :attr:`observation_space` - The Space object corresponding to valid observations, all valid observations should be contained within the space. - :attr:`spec` - An environment spec that contains the information used to initialize the environment from :meth:`gymnasium.make` - :attr:`metadata` - The metadata of the environment, e.g. `{"render_modes": ["rgb_array", "human"], "render_fps": 30}`. For Jax or Torch, this can be indicated to users with `"jax"=True` or `"torch"=True`. - :attr:`np_random` - The random number generator for the environment. This is automatically assigned during ``super().reset(seed=seed)`` and when assessing :attr:`np_random`. .. seealso:: For modifying or extending environments use the :class:`gymnasium.Wrapper` class Note: To get reproducible sampling of actions, a seed can be set with ``env.action_space.seed(123)``. Note: For strict type checking (e.g. mypy or pyright), :class:`Env` is a generic class with two parameterized types: ``ObsType`` and ``ActType``. The ``ObsType`` and ``ActType`` are the expected types of the observations and actions used in :meth:`reset` and :meth:`step`. The environment's :attr:`observation_space` and :attr:`action_space` should have type ``Space[ObsType]`` and ``Space[ActType]``, see a space's implementation to find its parameterized type. render_modesdict[str, Any]metadataN str | None render_modeEnvSpec | Nonespeczspaces.Space[ActType] action_spacezspaces.Space[ObsType]observation_spaceznp.random.Generator | None _np_random int | None_np_random_seedc[e)a Run one timestep of the environment's dynamics using the agent actions. When the end of an episode is reached (``terminated or truncated``), it is necessary to call :meth:`reset` to reset this environment's state for the next episode. .. versionchanged:: 0.26 The Step API was changed removing ``done`` in favor of ``terminated`` and ``truncated`` to make it clearer to users when the environment had terminated or truncated which is critical for reinforcement learning bootstrapping algorithms. Args: action (ActType): an action provided by the agent to update the environment state. Returns: observation (ObsType): An element of the environment's :attr:`observation_space` as the next observation due to the agent actions. An example is a numpy array containing the positions and velocities of the pole in CartPole. reward (SupportsFloat): The reward as a result of taking the action. terminated (bool): Whether the agent reaches the terminal state (as defined under the MDP of the task) which can be positive or negative. An example is reaching the goal state or moving into the lava from the Sutton and Barto Gridworld. If true, the user needs to call :meth:`reset`. truncated (bool): Whether the truncation condition outside the scope of the MDP is satisfied. Typically, this is a timelimit, but could also be used to indicate an agent physically going out of bounds. Can be used to end the episode prematurely before a terminal state is reached. If true, the user needs to call :meth:`reset`. info (dict): Contains auxiliary diagnostic information (helpful for debugging, learning, and logging). This might, for instance, contain: metrics that describe the agent's performance state, variables that are hidden from observations, or individual reward terms that are combined to produce the total reward. In OpenAI Gym z>>)rtype__name__idr4s r'__str__ Env.__str__sP 99 tDz**+:6 6tDz**+1TYY\\N"= =r*cU$)z+Support with-statement for the environment.r8r4s r' __enter__ Env.__enter__s r*c$UR5 g)zFSupport with-statement for the environment and closes the environment.F)r9)r%argss r'__exit__ Env.__exit__s r*c[X5$)z9Checks if the attribute `name` exists in the environment.)hasattrr%names r'has_wrapper_attrEnv.has_wrapper_attr  t""r*c[X5$)z/Gets the attribute `name` from the environment.)getattrrXs r'get_wrapper_attrEnv.get_wrapper_attrr\r*TforcecLU(d[X5(a [XU5 gg)zhSets the attribute `name` on the environment with `value`, see `Wrapper.set_wrapper_attr` for more info.TF)rWsetattr)r%rYrGrbs r'set_wrapper_attrEnv.set_wrapper_attrs GD'' D &r*rEr&rreturn9tuple[ObsType, SupportsFloat, bool, bool, dict[str, Any]])r,rr-dict[str, Any] | Nonerhztuple[ObsType, dict[str, Any]]rhz&RenderFrame | list[RenderFrame] | NonerhEnv[ObsType, ActType])rhintrhnp.random.GeneratorrGrp)rSrrYstrrhboolrYrsrhrrYrsrGrrbrtrhrt)rK __module__ __qualname____firstlineno____doc__r__annotations__rrrr r(r1r5r9propertyr<r?r/setterrMrPrTrZr_re__static_attributes__r8r*r'rrs#!H!/3Hn3"K"D.(',,.2J*1"&OZ&&"&" B&"V )- +L+L' +L ( +LZ "D  $ $ " " > ##HLr*rWrapperObsTypeWrapperActTypec\rSrSrSrS!SjrS"SjrSSS.S#SjjrS$SjrS r \ S%S j5r \ S&S j5r \ S'S j5r \S(S j5rS)SjrS*SjrSS.S+SjjrSrSr\S,Sj5r\ S-Sj5r\R.S.Sj5r\ S/Sj5r\R.S0Sj5r\ S1Sj5r\R.S2Sj5r\ S3Sj5r\ S4Sj5r\R.S5Sj5r\ S5rS rg)6Wrapperi aWraps a :class:`gymnasium.Env` to allow a modular transformation of the :meth:`step` and :meth:`reset` methods. This class is the base class of all wrappers to change the behavior of the underlying environment. Wrappers that inherit from this class can modify the :attr:`action_space`, :attr:`observation_space` and :attr:`metadata` attributes, without changing the underlying environment's attributes. Moreover, the behavior of the :meth:`step` and :meth:`reset` methods can be changed by these wrappers. Some attributes (:attr:`spec`, :attr:`render_mode`, :attr:`np_random`) will point back to the wrapper's environment (i.e. to the corresponding attributes of :attr:`env`). Note: If you inherit from :class:`Wrapper`, don't forget to call ``super().__init__(env)`` cXl[U[5(dS[U535eSUlSUlSUlSUlg)zWraps an environment to allow a modular transformation of the :meth:`step` and :meth:`reset` methods. Args: env: The environment to wrap z-Expected env to be a `gymnasium.Env` but got N)env isinstancerrJ _action_space_observation_space _metadata _cached_specr%rs r'__init__Wrapper.__init__2s]     G :49+ F G CGGK04,0r*c8URRU5$)z]Uses the :meth:`step` of the :attr:`env` that can be overwritten to change the returned data.)rr(r$s r'r( Wrapper.stepCsxx}}V$$r*Nr+c4URRXS9$)z^Uses the :meth:`reset` of the :attr:`env` that can be overwritten to change the returned data.r+)rr1r0s r'r1 Wrapper.resetIsxx~~4~99r*c6URR5$)z_Uses the :meth:`render` of the :attr:`env` that can be overwritten to change the returned data.)rr5r4s r'r5Wrapper.renderOsxx  r*c6URR5$)z#Closes the wrapper and :attr:`env`.)rr9r4s r'r9 Wrapper.closeSsxx~~r*c.URR$)z6Returns the base environment's :attr:`np_random_seed`.)rr?r4s r'r?Wrapper.np_random_seedWsxx&&&r*c.URR$)zReturns the base environment of the wrapper. This will be the bare :class:`gymnasium.Env` environment, underneath all layers of wrappers. )rr<r4s r'r<Wrapper.unwrapped\s xx!!!r*c:URb UR$URRnUb[U[5(a/[ US5nSU;a[ U5nURS5 OSnSSKJ n U"UR5URS[U5R3US9n[ U5nU=RU4- slXlU$![a/n[ R"R%SUS U35 SnAgSnAff=f) znReturns the :attr:`Env` :attr:`spec` attribute with the `WrapperSpec` if the wrapper inherits from `EzPickle`.N _saved_kwargsrrr:rY entry_pointkwargszAn exception occurred (z%) while copying the environment spec=)rrrrr r^rpopgymnasium.envs.registrationr class_namerwrJrKadditional_wrappers Exception gymnasiumloggerwarn)r%env_specrr wrapper_speces r'r Wrapper.specds    ($$ $88==  $ 566 7F?%f-FJJu% ?&__&#/qd1D1D0EFL #H-,,?,%   %%-aS0UV^U_`  s8!C!! D+%DDc lSSKJn U"UR5URSUR3US9$)z+Generates a `WrapperSpec` for the wrappers.rrrr)rrrrwrK)clsrrs r'rWrapper.wrapper_specs8 <!>>*!CLL>:  r*cZ[X5(agURRU5$)zGChecks if the given attribute is within the wrapper or its environment.T)rWrrZrXs r'rZWrapper.has_wrapper_attrs% 4  88,,T2 2r*c[X5(a [X5$URRU5$![a&n[ SUR 5SU<35UeSnAff=f)zGets an attribute from the wrapper and lower environments if `name` doesn't exist in this object. Args: name: The variable name to get Returns: The variable with name in wrapper or lower environments zwrapper z has no attribute N)rWr^rr_AttributeErrorr)r%rYrs r'r_Wrapper.get_wrapper_attrsm 4  4& & xx0066! $t011CD8L s8 A(!A##A(Trac[X5(a [XU5 gURRXSS9nU(agU(a [XU5 gg)arSets an attribute on this wrapper or lower environment if `name` is already defined. Args: name: The variable name value: The new variable value force: Whether to create the attribute on this wrapper if it does not exists on the lower environment instead of raising an exception Returns: If the variable has been set in this or a lower wrapper. TFra)rWrdrre)r%rYrGrb already_sets r'reWrapper.set_wrapper_attrsN 4   D &((33Du3MKE*r*cLS[U5RURS3$)zCReturns the wrapper name and the :attr:`env` representation string.rI>)rJrKrr4s r'rMWrapper.__str__s$4:&&'z33r*c[U5$)z1Returns the string representation of the wrapper.)rsr4s r'__repr__Wrapper.__repr__s 4yr*cUR$)z&Returns the class name of the wrapper.)rK)rs r'rWrapper.class_names||r*c`URcURR$UR$)zmReturn the :attr:`Env` :attr:`action_space` unless overwritten then the wrapper :attr:`action_space` is used.)rrrr4s r'rWrapper.action_spaces,    %88(( (!!!r*cXlgN)rr%spaces r'rrs"r*c`URcURR$UR$)zwReturn the :attr:`Env` :attr:`observation_space` unless overwritten then the wrapper :attr:`observation_space` is used.)rrrr4s r'rWrapper.observation_spaces,  " " *88-- -&&&r*cXlgr)rrs r'rrs"'r*c`URcURR$UR$)z)Returns the :attr:`Env` :attr:`metadata`.)rrrr4s r'rWrapper.metadatas( >> !88$$ $~~r*cXlgr)rrFs r'rrsr*c.URR$)z,Returns the :attr:`Env` :attr:`render_mode`.)rrr4s r'rWrapper.render_modesxx###r*c.URR$)z4Returns the :attr:`Env` :attr:`np_random` attribute.rr/r4s r'r/Wrapper.np_randomsxx!!!r*c$XRlgrrrFs r'r/rs "r*c[S5e)zThis code will never be run due to __getattr__ being called prior this. It seems that @property overwrites the variable (`_np_random`) meaning that __getattr__ gets called with the missing variable. zTCan't access `_np_random` of a wrapper, use `.unwrapped._np_random` or `.np_random`.)rr4s r'rWrapper._np_randoms  b  r*)rrrrrrrm)r&rrh@tuple[WrapperObsType, SupportsFloat, bool, bool, dict[str, Any]]r,rr-rjrhz%tuple[WrapperObsType, dict[str, Any]]rk)rhrrl)rhr)rrrhrrrrurv)rhrs)rhz4spaces.Space[ActType] | spaces.Space[WrapperActType])rzspaces.Space[WrapperActType])rhz4spaces.Space[ObsType] | spaces.Space[WrapperObsType])rzspaces.Space[WrapperObsType])rhr)rGr)rhrrorq)rKrwrxryrzrr(r1r5r9r|r?r<r classmethodrrZr_rerMrrrr}rrrr/rr~r8r*r'rr s 1"%$% I%%)4:!:3H: .: ! ''""##J  3&HL24" =""##' =''(( __$$""##  r*rc\\rSrSrSrS SjrSSS.S SjjrS SjrS SjrS r g)ObservationWrapperiaVModify observations from :meth:`Env.reset` and :meth:`Env.step` using :meth:`observation` function. If you would like to apply a function to only the observation before passing it to the learning code, you can simply inherit from :class:`ObservationWrapper` and overwrite the method :meth:`observation` to implement that transformation. The transformation defined in that method must be reflected by the :attr:`env` observation space. Otherwise, you need to specify the new observation space of the wrapper by setting :attr:`self.observation_space` in the :meth:`__init__` method of your wrapper. c.[RX5 g)zTConstructor for the observation wrapper. Args: env: Environment to be wrapped. Nrrrs r'rObservationWrapper.__init__ #r*Nr+c^URRXS9up4URU5U4$)zvModifies the :attr:`env` after calling :meth:`reset`, returning a modified observation using :meth:`self.observation`.r+)rr1 observation)r%r,r-obsinfos r'r1ObservationWrapper.reset%s0HHNNN> $d**r*chURRU5up#pEnURU5X4XV4$)zpModifies the :attr:`env` after calling :meth:`step` using :meth:`self.observation` on the returned observations.)rr(rr%r&rreward terminated truncatedrs r'r(ObservationWrapper.step,s6<@88==;P8 ZD ,f)QQr*c[e)z{Returns a modified observation. Args: observation: The :attr:`env` observation Returns: The modified observation r")r%rs r'rObservationWrapper.observation3 "!r*r8rr)r&rrhr)rrrhr) rKrwrxryrzrr1r(rr~r8r*r'rrsM$%)4+!+3H+ .+RR IR "r*rc>\rSrSrSrSSjrS SjrS SjrSrg) RewardWrapperi?aHSuperclass of wrappers that can modify the returning reward from a step. If you would like to apply a function to the reward that is returned by the base environment before passing it to learning code, you can simply inherit from :class:`RewardWrapper` and overwrite the method :meth:`reward` to implement that transformation. c.[RX5 g)zOConstructor for the Reward wrapper. Args: env: Environment to be wrapped. Nrrs r'rRewardWrapper.__init__Grr*chURRU5up#pEnX RU5XEU4$)zGModifies the :attr:`env` :meth:`step` reward using :meth:`self.reward`.)rr(rrs r'r(RewardWrapper.stepOs4<@88==;P8 ZDKK/LLr*c[e)zReturns a modified environment ``reward``. Args: reward: The :attr:`env` :meth:`step` reward Returns: The modified `reward` r")r%rs r'rRewardWrapper.rewardVrr*r8Nrrg)rrrhr) rKrwrxryrzrr(rr~r8r*r'rr?s)$MM BM "r*rc>\rSrSrSrSSjrS SjrS SjrSrg) ActionWrapperibuSuperclass of wrappers that can modify the action before :meth:`step`. If you would like to apply a function to the action before passing it to the base environment, you can simply inherit from :class:`ActionWrapper` and overwrite the method :meth:`action` to implement that transformation. The transformation defined in that method must take values in the base environment’s action space. However, its domain might differ from the original action space. In that case, you need to specify the new action space of the wrapper by setting :attr:`action_space` in the :meth:`__init__` method of your wrapper. Among others, Gymnasium provides the action wrappers :class:`gymnasium.wrappers.ClipAction` and :class:`gymnasium.wrappers.RescaleAction` for clipping and rescaling actions. c.[RX5 g)zOConstructor for the action wrapper. Args: env: Environment to be wrapped. Nrrs r'rActionWrapper.__init__prr*cVURRURU55$)z]Runs the :attr:`env` :meth:`env.step` using the modified ``action`` from :meth:`self.action`.)rr(r&r$s r'r(ActionWrapper.stepxs xx}}T[[011r*c[e)zReturns a modified action before :meth:`step` is called. Args: action: The original :meth:`step` actions Returns: The modified actions r"r$s r'r&ActionWrapper.action~rr*r8Nr)r&rrhri)r&rrhr) rKrwrxryrzrr(r&r~r8r*r'rrbs& $2$2 B2 "r*r)rz __future__rcopyrtypingrrrrr numpynprr gymnasium.utilsr r rr rrrrrrrrrrrr8r*r'rs]"FF:@ )  ) m$ B''7" #BJ)*)*p &' NNGW <=p f)"'7!JK)"X "GGWgw>? "F%"GG^WgEF%"r*