ni(">SrSSKJr SSKrSSKJr "SS\5rg)zExtends the environment by adding observation and action history. The implementation is a bit dirty import of the implementation in the experimental branch. )spacesN)MinitaurReactiveEnvc^\rSrSrSrSrSrSrSrSr SS /S S .r SU4S jjr U4S jr U4Sjr U4SjrU4SjrSrSr\S5rSrSrSrU=r$)MinitaurExtendedEnva<The 'extended' environment for Markovian property. This class implements to include prior actions and observations to the observation vector, thus making the environment "more" Markovian. This is especially useful for systems with latencies. Args: history_length: the length of the historic data history_include_actions: a flag for including actions as history history_include_states: a flag for including states as history include_state_difference: a flag for including the first-order differences as history include_second_state_difference: a flag for including the second-order state differences as history. include_base_position: a flag for including the base as observation, never_terminate: if this is on, the environment unconditionally never terminates. action_scale: the scale of actions, i g@human rgb_array2)z render.modeszvideo.frames_per_secondc  >XlXlX lX0lX@lXPlX`lXplXlXl [R"URS-UR45Ul[R"URS-S45Ul[R"URUR 45UlSUl[&[(U]V"S0U D6 [,R."SSUR0R25Ul[,R."[R4*[R4UR75R25Ul[;UR0R<5UR0l[;UR8R<5UR8lg)Nrrg?) _kwargs_history_length_history_include_actions_history_include_states_include_state_difference _include_second_state_difference_include_base_position_include_leg_model_never_terminate _action_scalenpzerosMAX_BUFFER_SIZEPARENT_OBSERVATION_DIM_past_parent_observations_past_motor_angles ACTION_DIM _past_actions_countersuperr__init__rBox action_spaceshapeinf_get_observationobservation_spacelenlowflat_dim) selfhistory_lengthhistory_include_actionshistory_include_statesinclude_state_differenceinclude_second_state_differenceinclude_base_positioninclude_leg_modelnever_terminate action_scalekwargs __class__s k/home/james-whalen/.local/lib/python3.13/site-packages/pybullet_envs/minitaur/envs/minitaur_extended_env.pyr&MinitaurExtendedEnv.__init__-s\L)$;!#9 %=",K)"7/+%%'XXt/C/Ca/G/3/J/J/L&MD" hh(<([[U] 5n[R"U5nXlXR UR'URRnX*SURUR'/n/n[UR5Hn[URU- S- S5nUR(aURUR U5 UR (dMaURUR"U5 M URn[SURS- 5[SURS- 5p//pUR$(a URUURU- /n UR&(a3URUSURU-- URU-/n /n UR((a.[R"URR+55n /n UR,(a+URR/5n UR1U 5n U/U-U-U -U -U /-U /-n[R2"U5nU$)z>Maybe concatenate motor velocity and torque into observations.Nrr)r%rr+rarray _observationr r$minitaur num_motorsr!rangermaxrappendrr#rrrGetBasePositionrGetMotorAnglesconvert_to_leg_model concatenate)r0parent_observationrDhistory_stateshistory_actionsittmtmmstate_differencesecond_state_difference base_position leg_modelraw_motor_anglesobservation_listfull_observationr;s r<r+$MinitaurExtendedEnv._get_observationSsI2DJL"45*4F""4==1))J-? -MDDMM*NO 4'' ( dmma!#Q 'a % %d<'>r'B B ,,  ! !! $q4+B+B2+F'F F  ! !# & '! M ""hh = = ?AmI 557++,<=i ~-? 2 36C_ E    ~~&67 r>c>SUlURUR5 [R"URS-UR 45Ul[R"URS-S45Ul[R"URUR45Ul SUl[R"[[U]755$)z#Resets the time and history buffer.rrr)r$_signalrrrrr r!r"r#rAr%rresetr0r;s r<r]MinitaurExtendedEnv.resetsDMLL%'XXt/C/Ca/G/3/J/J/L&MD" hh(<(c>XR-nXRUR'U=RS- sl[[U]U5up#pEUR 5nURUS9 X&XE4$)a Step function wrapper can be used to add shaping terms to the reward. Args: action: an array of the given action Returns: next_obs: the next observation reward: the reward for this experience tuple done: the terminal flag info: an additional information r) base_reward)rr#r$r%rsteprewardupdate)r0actionnext_obs_doneinforcr;s r<rbMinitaurExtendedEnv.stepsn    F(.t}}%MMQM#$7CFKH [[]FKKFK# T ''r>c*>[[U] 5 g)z1The helper function to terminate the environment.N)r%r_closer^s r< terminateMinitaurExtendedEnv.terminates t+-r>cUR(agURURR55nUSnUSnSnX$:dX4:agURS:agg)zDetermines whether the env is terminated or not. checks whether 1) the front leg is bent too much or 2) the time exceeds the manually set weights. Returns: terminal: the terminal flag whether the env is terminated or not Frr@g?Ti)rrJrCrIr$)r0rVswing0swing1maximum_swing_angles r< _termination MinitaurExtendedEnv._terminationse  ))$--*F*F*HII q\F q\F #v'C  }}  r>cURR5nURnUSURS- U- n[R "USS5nUR URS- nUR [URS- S5nUR [URS- S5nUSU-- U-n[R"[R"U55n Sn U SU-- n U S U --n U $) zCompute rewards for the given time step. It considers two terms: 1) forward velocity reward and 2) action acceleration penalty. Returns: reward: the computed reward. rg?rr@r rg?) rCrHcontrol_time_step_last_base_positionrclipr#r$rFmeanabs) r0current_base_positiondtvelocityvelocity_rewardre prev_actionprev_prev_actionaccaction_acceleration_penaltyrcs r<rcMinitaurExtendedEnv.rewards!MM99;  B%a(4+C+CA+FF"LHgghc2O     1 2F$$S):A%>?K))#dmma.?*CD 1{? "%5 5C"$''"&&+"6 F cO##F c///F Mr>cSn[R"US-5n[U5H4nUSU-SU-S-upESUS--S-XT- -nSXE--nXbU'XrX1-'M6 U$)zA helper function to convert motor angles to leg model. Args: motor_angles: raw motor angles: Returns: leg_angles: the leg pose model represented in swing and extension. r@rv)rrrE) motor_anglesnum_legs leg_anglesrOmotor1motor2swing extensions r<rJ(MinitaurExtendedEnv.convert_to_leg_modelsH(Q,'J 8_#AE!a%!)4nfQ!Vns"fo6e)im!*  r>cSUR0$)z)A helper get state function for pickling.r:)r)r0s r< __getstate__ MinitaurExtendedEnv.__getstate__s dll ##r>c.UR"S0USD6 g)z)A helper set state function for pickling.r:Nr)r&)r0states r< __setstate__ MinitaurExtendedEnv.__setstate__sMM$E(O$r>)rr$rrrrrrrrrrBr#r!r r(r,) rTFFFFFFrv)__name__ __module__ __qualname____firstlineno____doc__rr"rINIT_EXTENSION_POSINIT_SWING_POSmetadatar&r+r]rbrmrsrc staticmethodrJrr__static_attributes__ __classcell__)r;s@r<rrs&/*. ,!#(  '+&+(-/4%*!&$$FL2h >(2.04.$%%r>r)rgymrnumpyr1pybullet_envs.minitaur.envs.minitaur_reactive_envrrrr>r<rs$Qr%-r%r>