agent has. Normalized against the maximum observed agent stamina
of 8000.}
\item{\textbf{Frozen} [Boolean] Indicates if the agent is Frozen. Frozen status can
\item{\textbf{Frozen} [Boolean] Indicates if the agent is Frozen. Frozen status can
happen when being tackled by another player.}
happen when tackling or being tackled by another player.}
\item{\textbf{Colliding\_with\_ball} [Boolean] Indicates if the agent
\item{\textbf{Colliding\_with\_ball} [Boolean] Indicates the agent
is colliding with the ball.}
is colliding with the ball.}
\item{\textbf{Colliding\_with\_player} [Boolean] Indicates if the agent
\item{\textbf{Colliding\_with\_player} [Boolean] Indicates the agent
is colliding with another player.}
is colliding with another player.}
\item{\textbf{Colliding\_with\_post} [Boolean] Indicates if the agent
\item{\textbf{Colliding\_with\_post} [Boolean] Indicates the agent
is colliding with a goal post.}
is colliding with a goal post.}
\item{\textbf{Kickable} [Boolean] Indicates if the agent is able to
\item{\textbf{Kickable} [Boolean] Indicates the agent is able to
kick the ball.}
kick the ball.}
\itemrange{2}{\textbf{Goal Center} [Landmark] Center point between the goal posts.}
\itemrange{2}{\textbf{Goal Center} [Landmark] Center point between the goal posts.}
\itemrange{2}{\textbf{Goal Post Top} [Landmark] Top goal post.}
\itemrange{2}{\textbf{Goal Post Top} [Landmark] Top goal post.}
...
@@ -421,43 +457,44 @@ number of features is $58 + 8*\textrm{num\_teammates} +
...
@@ -421,43 +457,44 @@ number of features is $58 + 8*\textrm{num\_teammates} +
\itemrange{2}{\textbf{Penalty Box Top} [Landmark] Top corner of the penalty box.}
\itemrange{2}{\textbf{Penalty Box Top} [Landmark] Top corner of the penalty box.}
\itemrange{2}{\textbf{Penalty Box Bot} [Landmark] Bottom corner of the penalty box.}
\itemrange{2}{\textbf{Penalty Box Bot} [Landmark] Bottom corner of the penalty box.}
\itemrange{2}{\textbf{Center Field} [Landmark] The left middle point of the
\itemrange{2}{\textbf{Center Field} [Landmark] The left middle point of the
HFO play area. True center of the full-field.}
HFO play area.}
\itemrange{2}{\textbf{Corner Top Left} [Landmark] Top left corner HFO Playfield.}
\itemrange{2}{\textbf{Corner Top Left} [Landmark] Top left corner HFO Playfield.}
\itemrange{2}{\textbf{Corner Top Right} [Landmark] Top right corner HFO Playfield.}
\itemrange{2}{\textbf{Corner Top Right} [Landmark] Top right corner HFO Playfield.}
\itemrange{2}{\textbf{Corner Bot Right} [Landmark] Bot right corner HFO Playfield.}
\itemrange{2}{\textbf{Corner Bot Right} [Landmark] Bot right corner HFO Playfield.}
\itemrange{2}{\textbf{Corner Bot Left} [Landmark] Bot left corner HFO Playfield.}
\itemrange{2}{\textbf{Corner Bot Left} [Landmark] Bot left corner HFO Playfield.}
\item{\textbf{OOB Left Dist} [Distance] Distance to the nearest
\item{\textbf{OOB Left Dist} [Proximity] Proximity to the nearest
point of the left side of the HFO playable area. E.g. distance
point of the left side of the HFO playable area. E.g. distance
remaining before the agent goes out of bounds in left field.}
remaining before the agent goes out of bounds in left field.}
\item{\textbf{OOB Right Dist} [Distance] Distance remaining before
\item{\textbf{OOB Right Dist} [Proximity] Proximity to the right
the agent goes out of bounds in right field.}
field line.}
\item{\textbf{OOB Top Dist} [Distance] Distance remaining before
\item{\textbf{OOB Top Dist} [Proximity] Proximity to the top field line.}
the agent goes out of bounds in top field.}
\item{\textbf{OOB Bot Dist} [Proximity] Proximity to the bottom field line.}
\item{\textbf{OOB Bot Dist} [Distance] Distance remaining before
\item{\textbf{Ball Pos Valid} [Valid] Indicates the ball position estimate is valid.}
the agent goes out of bounds in bottom field.}
\itemrange{1}{\textbf{Ball Angle} [Angle] Agent's angle to the ball.}
\item{\textbf{Ball Pos Valid} [Valid] Indicates if the ball position estimate is valid.}
\item{\textbf{Ball Dist} [Proximity] Proximity to the ball.}
\itemrange{1}{\textbf{Ball Angle} [Angle] Angle to the ball from the agent's perspective.}
\item{\textbf{Ball Vel Valid} [Valid] Indicates the ball velocity estimate is valid.}
\item{\textbf{Ball Dist} [Distance] Distance to the ball.}
\item{\textbf{Ball Vel Mag} [Other] Magnitude of the ball's velocity.}
\item{\textbf{Ball Vel Valid} [Valid] Indicates if the ball velocity estimate is valid.}
\item{\textbf{Ball Vel Mag} [Other] Global magnitude of the ball velocity. Normalized against the observed maximum ball velocity, 3.0.}
\itemrange{1}{\textbf{Ball Vel Ang} [Angle] Global angle of ball velocity.}
\itemrange{1}{\textbf{Ball Vel Ang} [Angle] Global angle of ball velocity.}
\item [$8T$] {\textbf{Teammate Features} [Player] One teammate feature set (8 features) for each teammate active in HFO, sorted by proximity to the agent.}
\item [$8T$] {\textbf{Teammate Features} [Player] One teammate feature set (8 features) for each teammate active in HFO, sorted by proximity to the agent.}
\item [$8O$] {\textbf{Opponent Features} [Player] One opponent feature set (8 features) for each opponent present, sorted by proximity to the player.}
\item [$8O$] {\textbf{Opponent Features} [Player] One opponent feature set (8 features) for each opponent present, sorted by proximity to the player.}
\end{enumerate}
\end{enumerate}
\section{Action Space}
\section{Action Space}
The HFO domain provides support for both low-level primitive actions
The HFO domain provides support for both low-level primitive actions,
and high-level strategic actions. Basic, parameterized actions are
mid-level, and high-level strategic actions. Low-level, parameterized
provided for locomotion and kicking. Additionally high-level strategic
actions are provided for locomotion and kicking. Mid-level actions are
actions are available for moving, shooting, passing and
still parameterized by capture high level activities such as
dribbling. Control of the agent's head and gaze is not provided and
dribbling. Finally, high-level discrete, strategic actions are
follows Agent2D's default strategy. Both low and high level actions
available for moving, shooting, passing and dribbling. Control of the
are available through the same interface. It is the responsibility of
agent's head and gaze is not provided and follows Agent2D's default
the user to faithfully report which action spaces were used.
strategy. Low, medium, and high level actions are available through
the same interface. As the choice of action spaces greatly influences
the challenge of learning, it is the responsibility of the user to
faithfully report which action spaces were used.
\subsection{Low Level Actions}
\subsection{Low Level Actions}
\label{sec:low_level_actions}
\label{sec:low_level_actions}
\begin{itemize}
\begin{itemize}[noitemsep]
\item{\textbf{Dash}(power, degrees): Moves the agent with power [-100,
\item{\textbf{Dash}(power, degrees): Moves the agent with power [-100,
100] where negative values move backwards. The relative direction
100] where negative values move backwards. The relative direction
of movement is given in degrees and varies between [-180,180] with 0
of movement is given in degrees and varies between [-180,180] with 0
...
@@ -475,7 +512,7 @@ the user to faithfully report which action spaces were used.
...
@@ -475,7 +512,7 @@ the user to faithfully report which action spaces were used.
\subsection{Mid Level Actions}
\subsection{Mid Level Actions}
\label{sec:mid_level_actions}
\label{sec:mid_level_actions}
\begin{itemize}
\begin{itemize}[noitemsep]
\item{\textbf{Kick$\_$To}(target$_x$, target$_y$, speed): Kicks the
\item{\textbf{Kick$\_$To}(target$_x$, target$_y$, speed): Kicks the
ball to the specified target point with the desired speed. Valid
ball to the specified target point with the desired speed. Valid
values for target$_{x,y}\in[-1,1]$ and speed $\in[0,3]$.}
values for target$_{x,y}\in[-1,1]$ and speed $\in[0,3]$.}
...
@@ -493,7 +530,7 @@ the user to faithfully report which action spaces were used.
...
@@ -493,7 +530,7 @@ the user to faithfully report which action spaces were used.
\subsection{High Level Actions}
\subsection{High Level Actions}
\label{sec:high_level_actions}
\label{sec:high_level_actions}
\begin{itemize}
\begin{itemize}[noitemsep]
\item{\textbf{Move}(): Re-positions the agent according to the
\item{\textbf{Move}(): Re-positions the agent according to the
strategy given by Agent2D. The \textit{move} command works only when
strategy given by Agent2D. The \textit{move} command works only when
agent does not have the ball. If the agent has the ball, another
agent does not have the ball. If the agent has the ball, another
...
@@ -510,7 +547,7 @@ the user to faithfully report which action spaces were used.
...
@@ -510,7 +547,7 @@ the user to faithfully report which action spaces were used.
\end{itemize}
\end{itemize}
\subsection{Special Actions}
\subsection{Special Actions}
\begin{itemize}
\begin{itemize}[noitemsep]
\item{\textbf{NO-OP}: Indicates that the agent should take no action.}
\item{\textbf{NO-OP}: Indicates that the agent should take no action.}
\item{\textbf{Quit}: Indicates to the agent server that you wish to
\item{\textbf{Quit}: Indicates to the agent server that you wish to
terminate the HFO environment.}
terminate the HFO environment.}
...
@@ -522,7 +559,7 @@ New agents may be developed in C++ or Python. In Python, as long as
...
@@ -522,7 +559,7 @@ New agents may be developed in C++ or Python. In Python, as long as
the hfo interface has been installed, the agent needs only to
the hfo interface has been installed, the agent needs only to
\verb+from hfo import *+. In C++ it is necessary to
\verb+from hfo import *+. In C++ it is necessary to
\verb+#include <HFO.hpp>+ and also to link against the shared object
\verb+#include <HFO.hpp>+ and also to link against the shared object
