37 lines
3.1 KiB
Markdown
37 lines
3.1 KiB
Markdown
---
|
|
layout: single
|
|
title: "Aquarium MARL Environment"
|
|
categories: research
|
|
tags: multi-agent-reinforcement-learning MARL simulation emergence complex-systems
|
|
excerpt: "Aquarium: Open-source MARL environment for predator-prey studies."
|
|
header:
|
|
teaser: /assets/figures/20_aquarium.png
|
|
scholar_link: "https://scholar.google.de/citations?user=NODAd94AAAAJ&hl=en"
|
|
---
|
|
|
|
{:style="display:block; width:40%" .align-right}
|
|
|
|
The study of complex interactions using Multi-Agent Reinforcement Learning (MARL), particularly **predator-prey dynamics**, often requires specialized simulation environments. To streamline research and avoid redundant development efforts, we introduce **Aquarium**: a versatile, open-source MARL environment specifically designed for investigating predator-prey scenarios and related **emergent behaviors**.
|
|
|
|
Key Features of Aquarium:
|
|
|
|
* **Framework Integration:** Built upon and seamlessly integrates with the popular **PettingZoo API**, allowing researchers to readily apply existing MARL algorithm implementations (e.g., from Stable-Baselines3, RLlib).
|
|
* **Physics-Based Movement:** Simulates agent movement on a two-dimensional, continuous plane with edge-wrapping boundaries, incorporating basic physics for more realistic interactions.
|
|
* **High Customizability:** Offers extensive configuration options for:
|
|
* **Agent-Environment Interactions:** Observation spaces, action spaces, and reward functions can be tailored to specific research questions.
|
|
* **Environmental Parameters:** Key dynamics like agent speeds, prey reproduction rates, predator starvation mechanisms, sensor ranges, and more are fully adjustable.
|
|
* **Visualization & Recording:** Includes a resource-efficient visualizer and supports video recording of simulation runs, facilitating qualitative analysis and understanding of agent behaviors.
|
|
|
|
<div style="display: flex; align-items: center; justify-content: center;">
|
|
<center>
|
|
<img src="/assets/figures/20_observation_vector.png" alt="Diagram detailing the construction of the observation vector for an agent" style="display:inline-table; width:85%">
|
|
<figcaption>Construction details of the agent observation vector.</figcaption>
|
|
</center>
|
|
<center>
|
|
<img src="/assets/figures/20_capture_statistics.png" alt="Graphs showing average captures or rewards per prey agent under different training regimes" style="display:inline-table; width:100%">
|
|
<figcaption>Performance metrics (e.g., average captures/rewards) comparing training strategies.</figcaption>
|
|
</center>
|
|
</div>
|
|
|
|
To demonstrate its capabilities, we conducted preliminary studies using **Proximal Policy Optimization (PPO)** to train multiple prey agents learning to evade a predator within Aquarium. Consistent with findings in existing MARL literature, our results showed that training agents with **individual policies led to suboptimal performance**, whereas utilizing **parameter sharing** among prey agents significantly improved coordination, sample efficiency, and overall evasion success. {% cite kolle2024aquarium %}
|