hI, n S r SSKrSSKrSSKrSSKJr SSKJr SSK J
r
" S S\R 5 r g)a
@author: Olivier Sigaud
A merge between two sources:
* Adaptation of the MountainCar Environment from the "FAReinforcement" library
of Jose Antonio Martin H. (version 1.0), adapted by 'Tom Schaul, tom@idsia.ch'
and then modified by Arnaud de Broissia
* the gymnasium MountainCar environment
itself from
http://incompleteideas.net/sutton/book/MountainCar/MountainCar1.cp
permalink: https://perma.cc/6Z2N-PFWC
N)spaces)utils)DependencyNotInstalledc ^ \ rS rSrSrSS/SS.rSS\S- 4S jjrS
\R 4S jr
SSS .S
\ S- S\ S- 4U 4S jjjr
S rS rS rSrU =r$ )Continuous_MountainCarEnv a_
## Description
The Mountain Car MDP is a deterministic MDP that consists of a car placed stochastically
at the bottom of a sinusoidal valley, with the only possible actions being the accelerations
that can be applied to the car in either direction. The goal of the MDP is to strategically
accelerate the car to reach the goal state on top of the right hill. There are two versions
of the mountain car domain in gymnasium: one with discrete actions and one with continuous.
This version is the one with continuous actions.
This MDP first appeared in [Andrew Moore's PhD Thesis (1990)](https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-209.pdf)
```
@TECHREPORT{Moore90efficientmemory-based,
author = {Andrew William Moore},
title = {Efficient Memory-based Learning for Robot Control},
institution = {University of Cambridge},
year = {1990}
}
```
## Observation Space
The observation is a `ndarray` with shape `(2,)` where the elements correspond to the following:
| Num | Observation | Min | Max | Unit |
|-----|--------------------------------------|-------|------|---------------|
| 0 | position of the car along the x-axis | -1.2 | 0.6 | position (m) |
| 1 | velocity of the car | -0.07 | 0.07 | velocity (v) |
## Action Space
The action is a `ndarray` with shape `(1,)`, representing the directional force applied on the car.
The action is clipped in the range `[-1,1]` and multiplied by a power of 0.0015.
## Transition Dynamics:
Given an action, the mountain car follows the following transition dynamics:
*velocityt+1 = velocityt + force * self.power - 0.0025 * cos(3 * positiont)*
*positiont+1 = positiont + velocityt+1*
where force is the action clipped to the range `[-1,1]` and power is a constant 0.0015.
The collisions at either end are inelastic with the velocity set to 0 upon collision with the wall.
The position is clipped to the range [-1.2, 0.6] and velocity is clipped to the range [-0.07, 0.07].
## Reward
A negative reward of *-0.1 * action2* is received at each timestep to penalise for
taking actions of large magnitude. If the mountain car reaches the goal then a positive reward of +100
is added to the negative reward for that timestep.
## Starting State
The position of the car is assigned a uniform random value in `[-0.6 , -0.4]`.
The starting velocity of the car is always assigned to 0.
## Episode End
The episode ends if either of the following happens:
1. Termination: The position of the car is greater than or equal to 0.45 (the goal position on top of the right hill)
2. Truncation: The length of the episode is 999.
## Arguments
Continuous Mountain Car has two parameters for `gymnasium.make` with `render_mode` and `goal_velocity`.
On reset, the `options` parameter allows the user to change the bounds used to determine the new random state.
```python
>>> import gymnasium as gym
>>> env = gym.make("MountainCarContinuous-v0", render_mode="rgb_array", goal_velocity=0.1) # default goal_velocity=0
>>> env
>>>>
>>> env.reset(seed=123, options={"low": -0.7, "high": -0.5}) # default low=-0.6, high=-0.4
(array([-0.5635296, 0. ], dtype=float32), {})
```
## Version History
* v0: Initial versions release
human rgb_array ) render_modes
render_fpsN render_modec SU l SU l SU l SU l SU l SU l X l SU l [ R " U R U R * /[ R S9U l [ R " U R U R /[ R S9U l Xl
S U l S
U l S U l S U l S U l [& R( " U R U R S [ R S
9U l [& R( " U R U R [ R S9U l g )Ng g ?g333333g333333?gQ ??g~jtX?dtypeiX i T) )lowhighshaper )r r r )
min_action
max_action min_position max_position max_speed
goal_position
goal_velocitypowernparrayfloat32 low_state
high_stater screen_width
screen_heightscreenclockisopenr Box action_spaceobservation_space)selfr r s p/home/james-whalen/.local/lib/python3.13/site-packages/gymnasium/envs/classic_control/continuous_mountain_car.py__init__"Continuous_MountainCarEnv.__init__t s
+
0
((
/rzz
'
"JJdooT
"(T__BJJ"
actionc f U R S nU R S n[ [ US U R 5 U R 5 nX4U R
- S[ R " SU- 5 - -
-
nX0R : a U R nX0R * : a
U R * nX#-
nX R : a U R nX R : a U R nX R :X a US: a Sn[ X R : =( a X0R : 5 nSnU( a SnU[ R " US S5 S- - n[ R " X#/[ R" S9U l U R$ S :X a U R' 5 U R XeS
0 4$ ) Nr r g{Gzd? g Y@ g?r r F)stateminmaxr r r mathcosr r r boolr r powr r r! r render)r, r1 positionvelocityforce
terminatedrewards r- step Continuous_MountainCarEnv.step sp ::a=::a=Cq 4??3T__EDJJ&$((1x<2H)HHH nn $~~H ~~o % H '' '((H '' '((H (( (X\H ** * Mx;M;M/M
F$((6!9a(3..XXx 2"**E
w & KKMzz6ub88r0 )seedoptionsrD rE c X > [ TU ] US9 [ R " USS5 u p4[ R
" U R R X4S9S/5 U l U R S:X a U R 5 [ R
" U R [ R S90 4$ )N)rD g333333gٿ)r r r r r ) superresetr maybe_parse_reset_boundsr r np_randomuniformr5 r r< r! )r, rD rE r r __class__s r- rH Continuous_MountainCarEnv.reset s
4
227D$G XXt~~ 5 5# 5 I1 MN
w & KKMxx
"**5r99r0 c @ [ R " SU- 5 S- S- $ )Nr3 r g?)r sin)r, xss r- _height!Continuous_MountainCarEnv._height s vva"f~$t++r0 c
\ U R cG U R c e[ R R SU R R
S35 g SS KnSSKJn U R c UR 5 U R S:X aQ UR R 5 UR R U R U R 45 U l
O,UR! U R U R 45 U l
U R" c UR$ R' 5 U l U R( U R* -
nU R U- nSnSnUR! U R U R 45 U l U R, R/ S 5 U R0 S n[2 R4 " U R* U R( S
5 n U R7 U 5 n
[9 [; XR* -
U- X- 5 5 n UR< R? U R, U S S S
9 Sn U* S- US- US4u pnn/ nU
U4X4X4UU44 H nUR@ RC U5 RE [@ RF " SU- 5 5 nURI US XR* -
U- - US U - U R7 U5 U- - 45 M URK U R, US 5 URM U R, US 5 US- S4U* S- S44 H nUR@ RC U5 RE [@ RF " SU- 5 5 n[O US XR* -
U- - 5 [O US U - U R7 U5 U- - 5 4nURQ U R, US US [O US- 5 S5 URS U R, US US [O US- 5 S5 M [O U RT U R* -
U- 5 n[O U R7 U RT 5 U- 5 nUS- nURW U R, UUUS 5 URK U R, UU4UUS-
4US- US-
4/S5 URM U R, UU4UUS-
4US- US-
4/S5 URX R[ U R, S S5 U l U R R] U R, S5 U R S:X a] UR^ Ra 5 U R" Rc U Rd S 5 UR R[ 5 g U R S :X aL [2 Rf " [2 Rh " URj Rm U R 5 5 S S 9$ g ! [ a n[ S5 UeS nAff = f) NzYou are calling render method without specifying any render mode. You can specify the render_mode at initialization, e.g. gym.make("z", render_mode="rgb_array")r )gfxdrawzGpygame is not installed, run `pip install "gymnasium[classic_control]"`r ( ) rW rW d F)r r r )pointsclosedcolor
r4 r3 r g @) r^ r^ 2 ) rb r T)r r r
r
)r r r4 )axes)7r specgymloggerwarnidpygamerT ImportErrorr r& initdisplayset_moder$ r% Surfacer' timeClockr r surffillr5 r linspacerQ listzipdrawaalinesr8 Vector2
rotate_radr9 append aapolygonfilled_polygonintaacircle
filled_circler vline transformflipbliteventpumptickmetadata transposer surfarraypixels3d)r, ri rT e world_widthscalecarwidth carheightposrP ysxys clearancelrtbcoordscwheelflagxflagy1flagy2s r- r< Continuous_MountainCarEnv.render s~ #99( (( JJOO""&)),, /JL
& ;; KKM 7*##% $nn 5 5&&(:(:; %nnd.?.?ASAS-T U :: **,DJ''$*;*;; !!K/ NND$5$5t7I7I#JK 'jjm
[[**D,=,=s
C
\\"
3 . ..%7DE DIIc%yQ Y]HqL)Q>
aa&1&1&1a&1A ##A&11$((1s72CDA MMaDC"3"3 3u<<aD9$t||C'85'@@
2 $))VY7 tyy&)<a<#xi!mQ%78A ##A&11$((1s72CDAAaDC"3"3 3u<<=AaD9$t||C'85'@@AE
58U1Xs9s?/C_
! ! 58U1Xs9s?/C_
9 T''$*;*;;uDET\\$"4"45=>"
dii B IIV_ufrk 2URZ!4L M
IIV_ufrk 2URZ!4L M
$$))$))UDA F+ w & LL JJOODMM, 7 8 NN !
,<<))224;;?@y
-m (Y
s
X
X+ X&&X+c U R b6 SS KnUR R 5 UR 5 SU l g g )Nr F)r& ri rl quitr( )r, ri s r- close Continuous_MountainCarEnv.close/ s4 ;; " NN ! KKM DK #r0 )r* r' r r r# r( r" r r r r r r+ r r r& r% r$ r5 rq )Nr )__name__
__module__ __qualname____firstlineno____doc__r strr. r ndarrayrB r} dictrH rQ r< r __static_attributes__
__classcell__)rL s @r- r r s} Rj !+.H
C$J
D!92:: !9F +/t :S4Z : : :,fP r0 r )
r r8 numpyr gymnasiumre r gymnasium.envs.classic_controlr gymnasium.errorr Envr r0 r- r s0
0 2[ [ r0