steffen/website
1
0
Fork 0
Code Actions Packages Releases Activity

Website overhaul

Browse Source
This commit is contained in:
Steffen Illium
2025-03-27 22:57:31 +01:00
parent 2b75326eac
commit 755fd297bb
70 changed files with 1389 additions and 709 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
_posts/research/2024-01-13-aquarium-marl-environment.md Normal file
View File

@@ -0,0 +1,36 @@
---
layout: single
title: "Aquarium MARL Environment"
categories: research
tags: multi-agent-reinforcement-learning MARL simulation emergence complex-systems
excerpt: "Aquarium: Open-source MARL environment for predator-prey studies."
header:
teaser: /assets/figures/20_aquarium.png
scholar_link: "https://scholar.google.de/citations?user=NODAd94AAAAJ&hl=en"
---
![Diagram illustrating the multi-agent reinforcement learning cycle within the Aquarium environment](\assets\figures\20_aquarium.png){:style="display:block; width:40%" .align-right}
The study of complex interactions using Multi-Agent Reinforcement Learning (MARL), particularly **predator-prey dynamics**, often requires specialized simulation environments. To streamline research and avoid redundant development efforts, we introduce **Aquarium**: a versatile, open-source MARL environment specifically designed for investigating predator-prey scenarios and related **emergent behaviors**.
Key Features of Aquarium:
* **Framework Integration:** Built upon and seamlessly integrates with the popular **PettingZoo API**, allowing researchers to readily apply existing MARL algorithm implementations (e.g., from Stable-Baselines3, RLlib).
* **Physics-Based Movement:** Simulates agent movement on a two-dimensional, continuous plane with edge-wrapping boundaries, incorporating basic physics for more realistic interactions.
* **High Customizability:** Offers extensive configuration options for:
* **Agent-Environment Interactions:** Observation spaces, action spaces, and reward functions can be tailored to specific research questions.
* **Environmental Parameters:** Key dynamics like agent speeds, prey reproduction rates, predator starvation mechanisms, sensor ranges, and more are fully adjustable.
* **Visualization & Recording:** Includes a resource-efficient visualizer and supports video recording of simulation runs, facilitating qualitative analysis and understanding of agent behaviors.
<div style="display: flex; align-items: center; justify-content: center;">
<center>
<img src="/assets/figures/20_observation_vector.png" alt="Diagram detailing the construction of the observation vector for an agent" style="display:inline-table; width:85%">
<figcaption>Construction details of the agent observation vector.</figcaption>
</center>
<center>
<img src="/assets/figures/20_capture_statistics.png" alt="Graphs showing average captures or rewards per prey agent under different training regimes" style="display:inline-table; width:100%">
<figcaption>Performance metrics (e.g., average captures/rewards) comparing training strategies.</figcaption>
</center>
</div>
To demonstrate its capabilities, we conducted preliminary studies using **Proximal Policy Optimization (PPO)** to train multiple prey agents learning to evade a predator within Aquarium. Consistent with findings in existing MARL literature, our results showed that training agents with **individual policies led to suboptimal performance**, whereas utilizing **parameter sharing** among prey agents significantly improved coordination, sample efficiency, and overall evasion success. {% cite kolle2024aquarium %}