h`b S SK Jr S SKJr S SKJr S SKrS SKr S SKJ
r
J r J r S SK
Jr S SKJr / SQrS r " S
S \
5 rg) )closing)StringIO)pathN)Envspacesutils)categorical_sample)DependencyNotInstalled) +---------+z |R: | : :G|z | : | : : |z | : : : : |z | | : | : |z |Y| : |B: |r )i& i^ c ^ \ rS rSrSr/ SQSS.rS rS rS rS S jr S
r
S S \ S - S
\ S\ 4S jjr
S rS rS\4S jrS rS S S.S\S - S\S - 4U 4S jjjrS rS rS rS rS rS rU =r$ )!TaxiEnv u[
The Taxi Problem involves navigating to passengers in a grid world, picking them up and dropping them
off at one of four locations.
## Description
There are four designated pick-up and drop-off locations (Red, Green, Yellow and Blue) in the
5x5 grid world. The taxi starts off at a random square and the passenger at one of the
designated locations.
The goal is move the taxi to the passenger's location, pick up the passenger,
move to the passenger's desired destination, and
drop off the passenger. Once the passenger is dropped off, the episode ends.
The player receives positive rewards for successfully dropping-off the passenger at the correct
location. Negative rewards for incorrect attempts to pick-up/drop-off passenger and
for each step where another reward is not received.
Map:
+---------+
|R: | : :G|
| : | : : |
| : : : : |
| | : | : |
|Y| : |B: |
+---------+
From "Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition"
by Tom Dietterich [1].
## Action Space
The action shape is `(1,)` in the range `{0, 5}` indicating
which direction to move the taxi or to pickup/drop off passengers.
- 0: Move south (down)
- 1: Move north (up)
- 2: Move east (right)
- 3: Move west (left)
- 4: Pickup passenger
- 5: Drop off passenger
## Observation Space
There are 500 discrete states since there are 25 taxi positions, 5 possible
locations of the passenger (including the case when the passenger is in the
taxi), and 4 destination locations.
Destination on the map are represented with the first letter of the color.
Passenger locations:
- 0: Red
- 1: Green
- 2: Yellow
- 3: Blue
- 4: In taxi
Destinations:
- 0: Red
- 1: Green
- 2: Yellow
- 3: Blue
An observation is returned as an `int()` that encodes the corresponding state, calculated by
`((taxi_row * 5 + taxi_col) * 5 + passenger_location) * 4 + destination`
Note that there are 400 states that can actually be reached during an
episode. The missing states correspond to situations in which the passenger
is at the same location as their destination, as this typically signals the
end of an episode. Four additional states can be observed right after a
successful episodes, when both the passenger and the taxi are at the destination.
This gives a total of 404 reachable discrete states.
## Starting State
The initial state is sampled uniformly from the possible states
where the passenger is neither at their destination nor inside the taxi.
There are 300 possible initial states: 25 taxi positions, 4 passenger locations (excluding inside the taxi)
and 3 destinations (excluding the passenger's current location).
## Rewards
- -1 per step unless other reward is triggered.
- +20 delivering passenger.
- -10 executing "pickup" and "drop-off" actions illegally.
An action that results a noop, like moving into a wall, will incur the time step
penalty. Noops can be avoided by sampling the `action_mask` returned in `info`.
## Episode End
The episode ends if the following happens:
- Termination:
1. The taxi drops off the passenger.
- Truncation (when using the time_limit wrapper):
1. The length of the episode is 200.
## Information
`step()` and `reset()` return a dict with the following keys:
- p - transition probability for the state.
- action_mask - if actions will cause a transition to a new state.
For some cases, taking an action will have no effect on the state of the episode.
In v0.25.0, ``info["action_mask"]`` contains a np.ndarray for each of the actions specifying
if the action will change the state.
To sample a modifying action, use ``action = env.action_space.sample(info["action_mask"])``
Or with a Q-value based algorithm ``action = np.argmax(q_values[obs, np.where(info["action_mask"] == 1)[0]])``.
## Arguments
```python
import gymnasium as gym
gym.make('Taxi-v3')
```
`is_raining=False`: If True the cab will move in intended direction with
probability of 80% else will move in either left or right of target direction with
equal probability of 10% in both directions.
`fickle_passenger=False`: If true the passenger has a 30% chance of changing
destinations when the cab has moved one square away from the passenger's source location. Passenger fickleness
only happens on the first pickup and successful movement. If the passenger is dropped off at the source location
and picked up again, it is not triggered again.
## References
[1] T. G. Dietterich, “Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition,”
Journal of Artificial Intelligence Research, vol. 13, pp. 227–303, Nov. 2000, doi: 10.1613/jair.639.
## Version History
* v3: Map Correction + Cleaner Domain Description, v0.25.0 action masking added to the reset and step information
- In Gymnasium `1.2.0` the `is_rainy` and `fickle_passenger` arguments were added to align with Dietterich, 2000
* v2: Disallow Taxi start location = goal location, Update Taxi observations in the rollout, Update Taxi reward threshold.
* v1: Remove (3,2) from locs, add passidx<4 check
* v0: Initial version release
)humanansi rgb_array ) render_modes
render_fpsc N US: a XR U :X a SnUnXE4$ UnSnXE4$ )z7Computes the new location and reward for pickup action.r )locs)selftaxi_locpass_idxreward new_pass_idx
new_rewards V/home/james-whalen/.local/lib/python3.13/site-packages/gymnasium/envs/toy_text/taxi.py_pickupTaxiEnv._pickup sA aCalculates the new position for a row and col to the movement.r r, r- r. )r5 r3 r4 r7 r6 ) r r: r; movementoffsetdrdcr> r? s r _calc_new_positionTaxiEnv._calc_new_position sn aSXt||45aSXt||45 99Q[!g+"66 74 ?# #8Or! c U R XX45 nX4=n=pXUpn
Sn
SnSSSSS.nUS; dE US :X a U R S
U- S U- S - 4 S :X d! US :X a U R S
U- S U- 4 S :X a~ X S
u nn[ S
[ UU- U R 5 5 n
[ S
[ UU- U R
5 5 n U R
XX S
S S9nU R
XX S 5 n O6US:X a U R X|U
5 u pOUS:X a U R X|XM5 u pnU R XX5 nUS :: a U R US
US
X5 nU R U S
U S
X5 nU R U U R SUSU45 U R U U R SUSU45 U R U U R SUSU45 gU R U U R SUX45 g)z^Computes the next action for a state (row, col, pass_idx, dest_idx) and action for `is_rainy`.r+ F)r, r r r+ r r, )r+ r rM rN )rN rL rO )rM rL rO r r, r- r/ > r r, r- r, r. r/ r )rF r r0 g?g?r1 N) r2 r6 r5 r3 r4 r7 rI r r( r8 r9 )r r: r; r r% r< r= r left_pos right_posr> r? r r r@ movesrG rH intended_state
left_state right_states r _build_rainy_transitions TaxiEnv._build_rainy_transitions s8 Ch9+.*448),8,
))()
f ! !c'1s7Q;*> ?4 G! !c'1s7*: ;t C]1%FB!Sr4<< 89G!Sr4<< 89G..s q9IRS.TH //%-:JKI
q[#'<<#O L&
q[/3}}0 ,L* W|N Q; Xa[(1+|VJ ++! ilLK
FF5M& ! ( (#~r:)N O FF5M& ! ( (#z2z)J K FF5M& ! ( (#{B
)K L FF5M& ! ( (#~v)R Sr! N render_modeis_rainyfickle_passengerc p [ R " [ SS9U l / SQ=U l n/ SQU l SnSnSnUS-
U l US-
U l [ R " U5 U l Sn[ U5 V V
s0 s H n U [ U5 V
s0 s H o/ _M sn
_M" sn
n U l [ U5 H n [ U5 H n [ [ U5 S- 5 H n
[ [ U5 5 H| nU R XX5 n U
S : a X:w a U R U == S-
ss' [ U5 H6 n
U( a U R U U U
UU
5 M! U R U U U
UU
5 M8 M~ M M M U =R U R R! 5 - sl [" R$ " U5 U l [" R$ " U5 U l Xl X0l U R, =( a U R. R1 5 S
: U l S U l S U l [8 S U R R: S - [8 S U R R: S - 4U l S U l S U l S U l! S U l" S U l# S U l$ S U l% g s sn
f s sn
n f ) Ncdtype))r r )r r )r r )r r/ )) r r )r r` r )r` r` r )r r r` i r0 r, r 333333?r )&npasarrayMAPr6 r locs_colorsr4 r7 zerosinitial_state_distribranger8 lenr2 rW rB sumr Discrete action_spaceobservation_spacerY r[ np_randomrandom fickle_stepwindowclock WINDOW_SIZEshape cell_size taxi_imgstaxi_orientation
passenger_imgdestination_img median_horiz median_vertbackground_img)r rY rZ r[ r
num_statesnum_rows num_columns num_actionsr= r< r: r; r r% s r __init__TaxiEnv.__init__ sv JJs#. ;; DQ
!| "Q %'XXj%9" z*
*
U;-?@-?6BJ-?@ @*
?C [) %c$i!m 4H$)#d)$4 $ Ch I#a [ TU ] US9 [ U R U R 5 U l S U l U R =( a U R R 5 S: U l SU l
U R S:X a U R 5 [ U R
5 SU R U R
5 S.4$ )N)r rb r r r1 r )superresetr rh ro r r r[ rp rq rx rY r r r )r r r __class__s r r
TaxiEnv.reset s
4
#D$>$>O 00RT^^5J5J5Ls5R ! w & KKM466{S9I9I$&&9Q RRRr! c U R cG U R c e[ R R SU R R
S35 g U R S:X a U R
5 $ U R U R 5 $ )NzYou are calling render method without specifying any render mode. You can specify the render_mode at initialization, e.g. gym.make("z", render_mode="rgb_array")r )rY specgymloggerwarnid _render_text _render_gui)r s r r TaxiEnv.render s #99( (( JJOO""&)),, /JL
'$$& &##D$4$45 5r! c SS K nU R cv UR 5 UR
R
S5 US:X a% UR
R [ 5 U l O US:X a UR [ 5 U l U R c S5 eU R c UR R 5 U l
U R Gc [ R " [ R " [" 5 S5 [ R " [ R " [" 5 S5 [ R " [ R " [" 5 S 5 [ R " [ R " [" 5 S
5 /nU Vs/ s HB nUR$ R' UR( R+ U5 U R, 5 PMD snU l
U R. cs [ R " [ R " [" 5 S 5 nUR$ R' UR( R+ U5 U R, 5 U l U R0 c [ R " [ R " [" 5 S 5 nUR$ R' UR( R+ U5 U R, 5 U l U R0 R3 S
5 U R4 c [ R " [ R " [" 5 S5 [ R " [ R " [" 5 S5 [ R " [ R " [" 5 S5 /nU Vs/ s HB nUR$ R' UR( R+ U5 U R, 5 PMD snU l U R6 c [ R " [ R " [" 5 S5 [ R " [ R " [" 5 S5 [ R " [ R " [" 5 S5 /nU Vs/ s HB nUR$ R' UR( R+ U5 U R, 5 PMD snU l U R8 cs [ R " [ R " [" 5 S5 nUR$ R' UR( R+ U5 U R, 5 U l U R: n[= SUR> S 5 GH$ n[= SUR> S 5 GH nXR, S - XpR, S - 4n U R RA U R8 U 5 Xg U S:X a? US:X d XgS-
U S:w a+ U R RA U R6 S U 5 M Xg U S:X aN XvR> S S-
:X d XgS- U S:w a+ U R RA U R6 S U 5 M Xg U S:X a, U R RA U R6 S U 5 GM& Xg U S:X a@ US:X d Xg US-
S:w a, U R RA U R4 S U 5 GMq Xg U S:X aO XR> S S-
:X d Xg US- S:w a, U R RA U R4 S U 5 GM Xg U S:X d GM U R RA U R4 S U 5 GM GM' [C U RD U RF 5 Hy u pUR U R, 5 n U R3 S5 U RI U
5 U RK U 5 n U R RA XS U S S- 45 M{ U RM U RN 5 u pnnUS: aB U R RA U R. U RK U RD U 5 5 U RP S ; a U RP U l) U RK U RD U 5 nU RK X45 nUS US :: au U R RA U R0 US US U R, S S- -
45 U R RA U R U RR U5 OtU R RA U R U RR U5 U R RA U R0 US US U R, S S- -
45 US:X a] URT RW 5 UR
RY 5 U R R[ U R\ S 5 g US:X aL [^ R` " [^ Rb " URd Rg U R 5 5 S S 9$ g ! [ a n[ S5 UeS nAff = fs snf s snf s snf ) Nr z@pygame is not installed, run `pip install "gymnasium[toy-text]"`Taxir r z;Something went wrong with pygame. This should never happen.zimg/cab_front.pngzimg/cab_rear.pngzimg/cab_right.pngzimg/cab_left.pngzimg/passenger.pngz
img/hotel.png z img/gridworld_median_left.pngz img/gridworld_median_horiz.pngz img/gridworld_median_right.pngz img/gridworld_median_top.pngz img/gridworld_median_vert.pngz img/gridworld_median_bottom.pngzimg/taxi_background.pngr, |r- -
r rP r )r, r r- )axes)4pygame ImportErrorr
rr initdisplay set_captionset_modert Surfacers timeClockrw r joindirname__file__ transformscaleimageloadrv ry rz set_alphar{ r| r} r6 ri ru blitzipr rf fill get_surf_locr r r rx eventpumpupdatetickmetadatarc transposearray surfarraypixels3d)r moder e
file_names file_namer6 yxcellcolor
color_celllocr r r r% dest_loc
taxi_locations r r TaxiEnv._render_gui s] ;; KKM NN & &v .w $nn 5 5k B $ $nn[ 9
KK # I H I # :: **,DJ >> ! $,,x02EF $,,x02DE $,,x02EF $,,x02DE J ", !+I &&v||'8'8'CT^^T!+ DN % $,,x"8:MNI!'!1!1!7!7 !!),dnn"D ' $,,x"8/JI#)#3#3#9#9 !!),dnn$D
* *3 / $ $,,x02QR $,,x02RS $,,x02RSJ ",!!+I &&v||'8'8'CT^^T!+!D # $,,x02PQ $,,x02QR $,,x02STJ ", !+I &&v||'8'8'CT^^T!+ D & $,,x"8:STI"("2"2"8"8 !!),dnn#D yyq$**Q-(A1djjm,NN1 --q>>!3D/DE !4!4d;71:%16Ta%[^t5KKK$$T%5%5a%8$?WQZ4'A **dq5k!n.DKK$$T%5%5a%8$?WQZ4'KK$$T%5%5a%8$?WQZ4'Q!Vtwq1u~7MKK$$T%6%6q%94@WQZ4'A **dga!en.DKK$$T%6%6q%94@WQZ4'KK$$T%6%6q%94@% - )* tyy$*:*:;KD 7J % OOE "##D)C KK Za&#a&2+)> ? < 26TVV1D.Hh a< KK T / /1B1B499XCV1W X ??l *$(OOD !$$TYYx%89 ))8*>?
A;-* * KK $$!hqkDNN1,=,B BC
KK T^^D,A,A BM R KK T^^D,A,A BM R KK $$!hqkDNN1,=,B BC
7? LL NN ! ! # JJOODMM, 7 8
[
<<))224;;?@y
!w (R
2 *! s, j+ A k A k A k+
k5 kkc h US S- S- U R S - US S- U R S - 4$ )Nr, r- r )rv )r map_locs r r TaxiEnv.get_surf_locI sF
Q"dnnQ&77 AJN NN1 : r! c
U R R 5 R 5 n[ 5 nU VVs/ s H# o3 Vs/ s H oDR S5 PM snPM% nnnU R U R
5 u pgpS n
US: az [ R " USU- SU- S- SSS9USU- SU- S- ' U R U u p[ R " USU - SU - S- S SS
9USU - SU - S- ' O:[ R " U
" USU- SU- S- 5 S SS9USU- SU- S- ' U R U u p[ R " USU
- SU- S- S 5 USU
- SU- S- ' UR S
R U Vs/ s H nSR U5 PM sn5 S
- 5 U R b% UR S/ SQU R S35 OUR S
5 [ U5 UR 5 sS S S 5 $ s snf s snnf s snf ! , ( d f g = f)Nzutf-8c U S:X a S$ U $ )N r r )r s r ul TaxiEnv._render_text..ulU s s(3 ) )r! r r, r- yellowT) highlightblue)boldgreenmagenta
z ()SouthNorthEastWestPickupDropoffz)
)r6 copytolistr r r r colorizer writer r r getvalue)r r6 outfileliner] r r r r r% r pipjdidjr: s r r TaxiEnv._render_textN s: yy~~ &&(*=ABTT404a!40TB15TVV1D.H * a<27..AL!!h,"23X3CH a(lQ . / YYx(FB&+nnAF AFQJ'd'CBKB
# 38..3q8|$Q\A%567D3CH a(lQ . / 8$"'..QVQVaZ1H)"TAF AFQJ
dii =# =>EF ?? & MMMdoo^__bc
MM$
W
##%
; 1B* !>
s) H6H1 H6 H< I1H6
Ic | U R b/ SS KnUR R 5 UR 5 g g )Nr )rr r r quit)r r s r close
TaxiEnv.closer s- ;; " NN ! KKM #r! )r8 rm r} rv rs r6 rz r[ rq rh r r rf r7 r4 r{ r| rn ry rY r rw rx rr )r )NFF) __name__
__module__ __qualname____firstlineno____doc__r r r( rB rI rW strboolr r2 r r r r dictr r r r r r __static_attributes__
__classcell__)r s @r r
r
s EP 7H
(8"K6 .Td #' !& D#4ZD# D# D#L
* TD # S DjS S S 6 AF
"&H r! r
)
contextlibr ior osr numpyrc gymnasiumr r r r gymnasium.envs.toy_text.utilsr gymnasium.errorr
re rt r
r r! r r$ s< ( ( < 2 ^ c ^ r!