added aquarium and mas emergence

_posts/research/2024-01-13-aquarium.md

@@ -0,0 +1,18 @@
+---
+layout: single
+title:  "Aquarium"
+categories: research MARL reinforcement-learning multi-agent
+excerpt: "Exploring Predator-Prey Dynamics"
+header:
+  teaser: assets/figures/20_aquarium.png
+---
+
+![Multi-Agent Reinforcement Learning Cycle](\assets\figures\20_aquarium.png){:style="display:block; width:40%" .align-right}
+Recent advances in multi-agent reinforcement learning have enabled the modeling of complex interactions between agents in simulated environments. In particular, predator-prey dynamics have garnered significant interest, and various simulations have been adapted to meet unique requirements. To avoid further time-intensive development efforts, we introduce *Aquarium*, a versatile multi-agent reinforcement learning environment designed for studying predator-prey interactions and emergent behavior. *Aquarium* is open-source and seamlessly integrates with the PettingZoo framework, allowing for a quick start using established algorithm implementations. It features physics-based agent movement on a two-dimensional, edge-wrapping plane. Both the agent-environment interactions (observations, actions, rewards) and environmental parameters (agent speed, prey reproduction, predator starvation, and more) are fully customizable. In addition to providing a resource-efficient visualization, *Aquarium* supports video recording, facilitating a visual understanding of agent behavior. 
+
+To showcase the environment's capabilities, we conducted preliminary studies using proximal policy optimization (PPO) to train multiple prey agents to evade a predator. Consistent with existing literature, we found that individual learning leads to worse performance, while parameter sharing significantly improves coordination and sample efficiency.
+{% cite kolle2024aquarium %}
+
+![Construction of the Observation Vector](\assets\figures\20_capture_statistics.png){:style="display:block; width:70%" .align-center}
+
+![Average captures and rewards per prey agent](\assets\figures\20_observation_vector.png){:style="display:block; width:70%" .align-center}

_posts/research/2024-10-27-emergence-mas.md

@@ -0,0 +1,18 @@
+---
+layout: single
+title:  "MAS Emergence"
+categories: research multi-agent reinforcement-learning safety emergence 
+excerpt: "A Safety Perspective"
+header:
+  teaser: assets/figures/21_coins_teaser.png
+---
+
+![Evaluation Environments](\assets\figures\21_envs.png){:style="display:block; width:40%" .align-right}
+Emergent effects can occur in multi-agent systems (MAS), where decision-making is decentralized and based on local information. These effects may range from minor deviations in behavior to catastrophic system failures. To formally define these phenomena, we identify misalignments between the global inherent specification (the true specification) and its local approximation (e.g., the configuration of distinct reward components or observations). Leveraging established safety concepts, we develop a framework for understanding these emergent effects. To demonstrate the resulting implications, we examine two highly configurable gridworld scenarios, where inadequate specifications lead to unintended behavior deviations when derived independently. Acknowledging that a global solution may not always be practical, we propose adjusting the underlying parameterizations to mitigate these issues, thereby improving system alignment and reducing the risk of emergent failures.
+{% cite altmann2024emergence %}
+
+![Instances of emergent behavior](\assets\figures\21_coins.png){:style="display:block; width:70%" .align-center}
+
+![Blocking behavior](\assets\figures\21_blocking.png){:style="display:block; width:70%" .align-center}
+
+