How can I access additional information about my gymnasium environment during a torchrl rollout?
June 07, 2025
To access additional information about your gymnasium environment during a torchrl rollout, you can use the
env_info dictionary. This dictionary contains a variety of information about the environment, including:
* Observation: The current observation from the environment.
* Reward: The reward received for the last action.
* Done: A flag indicating whether the episode has ended.
* Info: A dictionary of additional information about the environment, such as the current position of the agent.
To access the env_info dictionary, you can use the following code:
env_info = rollout_worker.env_info
The env_info dictionary can be used to get a variety of information about the environment. For example, the following code gets the current position of the agent:
position = env_info["position"]
You can also use the env_info dictionary to get information about the environment's state. For example, the following code gets the current velocity of the agent:
velocity = env_info["velocity"]
The env_info dictionary can be a valuable tool for debugging and understanding the behavior of your environment. It can also be used to extract additional features from the environment that can be used to improve the performance of your agent.
Here is an example of how you can use the env_info dictionary to improve the performance of your agent. In this example, we are using the MountainCar environment. The goal of the MountainCar environment is to drive a car up a hill.
The following code defines a simple agent that uses the env_info dictionary to get the current position of the car:
class SimpleAgent(nn.Module):
def __init__(self):
super(SimpleAgent, self).__init__()
def forward(self, env_info):
position = env_info["position"]
if position[0] < 0.5:
return 1 # Accelerate right
else:
return 2 # Accelerate left
The following code trains the agent using the torchrl library:
agent = SimpleAgent()
trainer = torchrl.Trainer(agent, env="MountainCar-v0")
trainer.train()
The following code evaluates the agent:
agent = SimpleAgent()
env = gym.make("MountainCar-v0")
env.render()
done = False
while not done:
env_info = env.reset()
while not done:
action = agent(env_info)
env_info, reward, done, _ = env.step(action)
env.render()
The agent is able to learn to drive the car up the hill using the env_info dictionary to get the current position of the car. This shows how the env_info dictionary can be used to improve the performance of your agent.