---
layout: single
title: "AI-Fusion Safety"
categories: projects
tags: multi-agent-systems reinforcement-learning safety emergence simulation
excerpt: "Studied MARL emergence and safety, built simulations with Fraunhofer."
header:
  teaser: /assets/images/projects/robot.png
role: Researcher, Software Developer
skills: Multi-Agent Reinforcement Learning (MARL), Emergence Analysis, AI Safety, Simulation Environment Design, Python, Gymnasium API, Software Engineering, Unity (Visualization), Industry Collaboration
---

<div class="container">
  <div class="sidebar" style="float: right; width: 25%; border: 0.5px grey solid; padding: 15px;">
  <h4 style="margin-top: 0;">Project Resources</h4>
  <ul style="list-style: none; padding-left: 0;">
    <li><a href="https://github.com/illiumst/marl-factory-grid/" target="_blank" rel="noopener noreferrer"><i class="fab fa-fw fa-github" aria-hidden="true"></i> GitHub Repo</a></li>
    <li><a href="https://pypi.org/project/Marl-Factory-Grid/" target="_blank" rel="noopener noreferrer"><i class="fab fa-fw fa-python" aria-hidden="true"></i> Install via PyPI</a></li>
    <li><a href="https://marl-factory-grid.readthedocs.io/en/latest/" target="_blank" rel="noopener noreferrer"><i class="fas fa-fw fa-book" aria-hidden="true"></i> ReadTheDocs</a></li>
    <li><i class="fas fa-fw fa-file-alt" aria-hidden="true"></i> {% cite altmann2024emergence %}</li>
  </ul>
  ![logo](\assets\images\projects\full_domain.png){: style="margin:0em; padding:0em; width:15em"}
  </div>
  <div class="main-content" style="float: left; width: 75%;">
    
  ![Robot Arm Icon](/assets/images/projects/robot.png){: .align-left style="padding:0.1em; width:5em"}
  **Project:** AI-Fusion<br>
  **Partner:** [Fraunhofer Institute for Cognitive Systems (IKS)](https://www.iks.fraunhofer.de/)<br>
  **Duration:** 2022 - 2023<br>
  **Objective:** To investigate the detection and mitigation of potentially unsafe emergent behaviors in complex systems composed of multiple interacting AI agents, particularly in scenarios involving heterogeneous agents (e.g., mixed-vendor autonomous systems).

  In collaboration with Fraunhofer IKS, the AI-Fusion project addressed the critical challenge of understanding and ensuring safety in multi-agent reinforcement learning (MARL) systems. Emergence, defined as the arising of complex, often unpredictable, system-level dynamics from local interactions between agents and their environment, was a central focus due to its implications for system safety and reliability.
  </div>
</div>

---

To facilitate research into these phenomena, key contributions included the development of specialized simulation tools:

**1. High-Performance MARL Simulation Environment:**

*   A flexible and efficient simulation environment was developed in Python, adhering to the [Gymnasium (formerly Gym) API specification](https://gymnasium.farama.org/main/).
*   **Purpose:** Designed specifically for training and evaluating reinforcement learning algorithms in multi-agent contexts prone to emergent behaviors.
*   **Features:**
    *   **Modularity:** Supports diverse scenarios through configurable `modules` and `configurations`.
    *   **Observation/Action Spaces:** Handles complex agent interactions, including per-agent observations and sequential/multi-agent action coordination.
    *   **Performance:** Optimized for efficient simulation runs, enabling extensive experimentation.

**2. Unity-Based Demonstrator Unit:**

*   A complementary visualization tool was created using the Unity engine.
*   **Purpose:** Allows for the replay, inspection, and detailed analysis of specific simulation scenarios and agent interactions.
*   **Utility:** Aids researchers in identifying and understanding the mechanisms behind observed emergent dynamics.
*   [View Demonstrator on GitHub](https://github.com/illiumst/F-IKS_demonstrator)

<div style="clear: both;"></div>


<center>
  <img src="/assets/images/projects/rel_emergence.png" alt="Diagram illustrating the concept of emergence from interactions between agents and environment" style="padding:0.1em; width:80%">
  <figcaption>Conceptual relationship defining emergence in multi-agent systems.</figcaption>
</center>


This project involved close collaboration with industry-focused researchers, software development adhering to modern standards, and deep investigation into the theoretical underpinnings of emergence and safety in MARL systems. The developed tools provide a valuable platform for continued research in this critical area.

{% include reference.html %}