---
layout: single
title:  "Aquarium"
categories: research MARL reinforcement-learning multi-agent
excerpt: "Exploring Predator-Prey Dynamics in multi-agent reinforcement-learning"
header:
  teaser: assets/figures/20_aquarium.png
---

![Multi-Agent Reinforcement Learning Cycle](\assets\figures\20_aquarium.png){:style="display:block; width:40%" .align-right}
Recent advances in multi-agent reinforcement learning have enabled the modeling of complex interactions between agents in simulated environments. In particular, predator-prey dynamics have garnered significant interest, and various simulations have been adapted to meet unique requirements. To avoid further time-intensive development efforts, we introduce *Aquarium*, a versatile multi-agent reinforcement learning environment designed for studying predator-prey interactions and emergent behavior. *Aquarium* is open-source and seamlessly integrates with the PettingZoo framework, allowing for a quick start using established algorithm implementations. It features physics-based agent movement on a two-dimensional, edge-wrapping plane. Both the agent-environment interactions (observations, actions, rewards) and environmental parameters (agent speed, prey reproduction, predator starvation, and more) are fully customizable. In addition to providing a resource-efficient visualization, *Aquarium* supports video recording, facilitating a visual understanding of agent behavior. 

To showcase the environment's capabilities, we conducted preliminary studies using proximal policy optimization (PPO) to train multiple prey agents to evade a predator. Consistent with existing literature, we found that individual learning leads to worse performance, while parameter sharing significantly improves coordination and sample efficiency.
{% cite kolle2024aquarium %}

![Construction of the Observation Vector](\assets\figures\20_capture_statistics.png){:style="display:block; width:70%" .align-center}

![Average captures and rewards per prey agent](\assets\figures\20_observation_vector.png){:style="display:block; width:70%" .align-center}