h~( n S r SSKrSSKrSSKrSSKJr SSKJr SSK J
r
" S S\R 5 r g)z^
http://incompleteideas.net/MountainCar/MountainCar1.cp
permalink: https://perma.cc/6Z2N-PFWC
N)spaces)utils)DependencyNotInstalledc ^ \ rS rSrSrSS/SS.rSS\S- 4S jjrS
\4S jr SSS .S
\S- S\
S- 4U 4S jjjr S r S r
S rS rSrU =r$ )MountainCarEnv a
## Description
The Mountain Car MDP is a deterministic MDP that consists of a car placed stochastically
at the bottom of a sinusoidal valley, with the only possible actions being the accelerations
that can be applied to the car in either direction. The goal of the MDP is to strategically
accelerate the car to reach the goal state on top of the right hill. There are two versions
of the mountain car domain in gymnasium: one with discrete actions and one with continuous.
This version is the one with discrete actions.
This MDP first appeared in [Andrew Moore's PhD Thesis (1990)](https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-209.pdf)
```
@TECHREPORT{Moore90efficientmemory-based,
author = {Andrew William Moore},
title = {Efficient Memory-based Learning for Robot Control},
institution = {University of Cambridge},
year = {1990}
}
```
## Observation Space
The observation is a `ndarray` with shape `(2,)` where the elements correspond to the following:
| Num | Observation | Min | Max | Unit |
|-----|--------------------------------------|-------|------|--------------|
| 0 | position of the car along the x-axis | -1.2 | 0.6 | position (m) |
| 1 | velocity of the car | -0.07 | 0.07 | velocity (v) |
## Action Space
There are 3 discrete deterministic actions:
- 0: Accelerate to the left
- 1: Don't accelerate
- 2: Accelerate to the right
## Transition Dynamics:
Given an action, the mountain car follows the following transition dynamics:
*velocityt+1 = velocityt + (action - 1) * force - cos(3 * positiont) * gravity*
*positiont+1 = positiont + velocityt+1*
where force = 0.001 and gravity = 0.0025. The collisions at either end are inelastic with the velocity set to 0
upon collision with the wall. The position is clipped to the range `[-1.2, 0.6]` and
velocity is clipped to the range `[-0.07, 0.07]`.
## Reward:
The goal is to reach the flag placed on top of the right hill as quickly as possible, as such the agent is
penalised with a reward of -1 for each timestep.
## Starting State
The position of the car is assigned a uniform random value in *[-0.6 , -0.4]*.
The starting velocity of the car is always assigned to 0.
## Episode End
The episode ends if either of the following happens:
1. Termination: The position of the car is greater than or equal to 0.5 (the goal position on top of the right hill)
2. Truncation: The length of the episode is 200.
## Arguments
Mountain Car has two parameters for `gymnasium.make` with `render_mode` and `goal_velocity`.
On reset, the `options` parameter allows the user to change the bounds used to determine the new random state.
```python
>>> import gymnasium as gym
>>> env = gym.make("MountainCar-v0", render_mode="rgb_array", goal_velocity=0.1) # default goal_velocity=0
>>> env
>>>>
>>> env.reset(seed=123, options={"x_init": np.pi/2, "y_init": 0.5}) # default x_init=np.pi, y_init=1.0
(array([-0.46352962, 0. ], dtype=float32), {})
```
## Version History
* v0: Initial versions release
human rgb_array ) render_modes
render_fpsN render_modec ` SU l SU l SU l SU l X l SU l SU l [ R " U R U R * /[ R S9U l
[ R " U R U R /[ R S9U l Xl SU l
S U l S U l S U l S
U l [$ R& " S 5 U l [$ R* " U R U R [ R S9U l g ) Ng333333g333333?gQ ?g ?gMbP?g{Gzd?dtypeiX i T ) min_position max_position max_speed
goal_position
goal_velocityforcegravitynparrayfloat32lowhighr screen_width
screen_heightscreenclockisopenr Discrete action_spaceBoxobservation_space)selfr r s e/home/james-whalen/.local/lib/python3.13/site-packages/gymnasium/envs/classic_control/mountain_car.py__init__MountainCarEnv.__init__l s *
88T . . @
SHHd / / @
S &
"OOA .!'DHHdiirzz!R actionc U R R U5 ( d U< S[ U5 S35 eU R u p#X1S-
U R - [
R " SU- 5 U R * - - -
n[ R " X0R * U R 5 nX#-
n[ R " X R U R 5 nX R :X a US: a Sn[ X R : =( a X0R : 5 nSnX#4U l U R S:X a U R# 5 [ R$ " U R [ R& S9XTS 0 4$ )
Nz (z ) invalid r r g r r F)r% containstypestater mathcosr r clipr r r boolr r r renderr r )r( r- positionvelocity
terminatedrewards r) stepMountainCarEnv.step sE ))
2Zr$v, y 1 2
"ZZaZ4::-X 0F4<<-0XXX778nn_dnnE 778%6%68I8IJ (( (X\H ** * Mx;M;M/M
)
w & KKMxx
"**5v5RTTTr, )seedoptionsr> r? c X > [ TU ] US9 [ R " USS5 u p4[ R
" U R R X4S9S/5 U l U R S:X a U R 5 [ R
" U R [ R S90 4$ )N)r> g333333gٿ)r r r r r ) superresetr maybe_parse_reset_boundsr r np_randomuniformr2 r r7 r )r( r> r? r r __class__s r) rB MountainCarEnv.reset s
4
227D$G XXt~~ 5 5# 5 I1 MN
w & KKMxx
"**5r99r, c @ [ R " SU- 5 S- S- $ )Nr g?g?)r sin)r( xss r) _heightMountainCarEnv._height s vva"f~$t++r, c
\ U R cG U R c e[ R R SU R R
S35 g SS KnSSKJn U R c UR 5 U R S:X aQ UR R 5 UR R U R U R 45 U l
O,UR! U R U R 45 U l
U R" c UR$ R' 5 U l U R( U R* -
nU R U- nSnSnUR! U R U R 45 U l U R, R/ S 5 U R0 S n[2 R4 " U R* U R( S
5 n U R7 U 5 n
[9 [; XR* -
U- X- 5 5 n UR< R? U R, U S S S
9 Sn U* S- US- US4u pnn/ nU
U4X4X4UU44 H nUR@ RC U5 RE [@ RF " SU- 5 5 nURI US XR* -
U- - US U - U R7 U5 U- - 45 M URK U R, US 5 URM U R, US 5 US- S4U* S- S44 H nUR@ RC U5 RE [@ RF " SU- 5 5 n[O US XR* -
U- - 5 [O US U - U R7 U5 U- - 5 4nURQ U R, US US [O US- 5 S5 URS U R, US US [O US- 5 S5 M [O U RT U R* -
U- 5 n[O U R7 U RT 5 U- 5 nUS- nURW U R, UUUS 5 URK U R, UU4UUS-
4US- US-
4/S5 URM U R, UU4UUS-
4US- US-
4/S5 URX R[ U R, S S5 U l U R R] U R, S5 U R S:X a] UR^ Ra 5 U R" Rc U Rd S 5 UR R[ 5 g U R S :X aL [2 Rf " [2 Rh " URj Rm U R 5 5 S S 9$ g ! [ a n[ S5 UeS nAff = f) NzYou are calling render method without specifying any render mode. You can specify the render_mode at initialization, e.g. gym.make("z", render_mode="rgb_array")r )gfxdrawzGpygame is not installed, run `pip install "gymnasium[classic_control]"`r ( ) rQ rQ d F)r r r )pointsclosedcolor
r r/ g @) rY rY 2 ) r] r T)r r r
r
)r/ r rW )axes)7r specgymloggerwarnidpygamerN ImportErrorr r! initdisplayset_moder r Surfacer" timeClockr r surffillr2 r linspacerK listzipdrawaalinesr3 Vector2
rotate_radr4 append aapolygonfilled_polygonintaacircle
filled_circler vline transformflipbliteventpumptickmetadata transposer surfarraypixels3d)r( rd rN e world_widthscalecarwidth carheightposrJ ysxys clearancelrtbcoordscwheelflagxflagy1flagy2s r) r7 MountainCarEnv.render s~ #99( (( JJOO""&)),, /JL
& ;; KKM 7*##% $nn 5 5&&(:(:; %nnd.?.?ASAS-T U :: **,DJ''$*;*;; !!K/ NND$5$5t7I7I#JK 'jjm
[[**D,=,=s
C
\\"
3 . ..%7DE DIIc%yQ Y]HqL)Q>
aa&1&1&1a&1A ##A&11$((1s72CDA MMaDC"3"3 3u<<aD9$t||C'85'@@
2 $))VY7 tyy&)<a<#xi!mQ%78A ##A&11$((1s72CDAAaDC"3"3 3u<<=AaD9$t||C'85'@@AE
58U1Xs9s?/C_
! ! 58U1Xs9s?/C_
9 T''$*;*;;uDET\\$"4"45=>"
dii B IIV_ufrk 2URZ!4L M
IIV_ufrk 2URZ!4L M
$$))$))UDA F+ w & LL JJOODMM, 7 8 NN !
,<<))224;;?@y
-m (Y
s
X
X+ X&&X+c SSSSS.$ )Nr/ r rW ) ) ) )r r r )r( s r) get_keys_to_action!MountainCarEnv.get_keys_to_action s q!;;r, c U R b6 SS KnUR R 5 UR 5 SU l g g )Nr F)r! rd rg quitr# )r( rd s r) closeMountainCarEnv.close s4 ;; " NN ! KKM DK #r, )r% r" r r r r r r# r r r r r' r r! r r r2 rl )Nr )__name__
__module__ __qualname____firstlineno____doc__r strr* rx r< dictrB rK r7 r r __static_attributes__
__classcell__)rF s @r) r r s Tn !+.H
SC$J S0U3 U6 # : Dj: : : ,fP< r, r )
r r3 numpyr gymnasiumr` r gymnasium.envs.classic_controlr gymnasium.errorr Envr r r, r) r s0
0 2Q SWW Q r,