---
title: "AI-Fusion Safety"
tags: [MARL, reinforcement-learning, safety, emergence, simulation]
excerpt: "Studied MARL emergence and safety, built simulations with Fraunhofer."
teaser: /images/projects/robot.png
icon: /images/projects/robot.png
role: Researcher, Software Developer
skills: Multi-Agent Reinforcement Learning (MARL), Emergence Analysis, AI Safety, Simulation Environment Design, Python, Gymnasium API, Software Engineering, Unity (Visualization), Industry Collaboration
---

In collaboration with Fraunhofer IKS, the AI-Fusion project addressed the critical challenge of understanding and ensuring safety in multi-agent reinforcement learning (MARL) systems. Emergence, defined as the arising of complex, often unpredictable, system-level dynamics from local interactions between agents and their environment, was a central focus due to its implications for system safety and reliability. The project's objective was to investigate the detection and mitigation of potentially unsafe emergent behaviors in complex systems composed of multiple interacting AI agents, particularly in scenarios involving heterogeneous agents (e.g., mixed-vendor autonomous systems).

<InfoBox title="Project Details">
  {/* Section 2: Project Info */}
  <h4>Overview</h4>
  <p style={{ lineHeight: '1.5', margin: 0 }}>
    <strong>Project:</strong> AI-Fusion<br/>
    <strong>Partner:</strong> <a href="https://www.iks.fraunhofer.de/" target="_blank" rel="noopener noreferrer">Fraunhofer IKS</a><br/>
    <strong>Duration:</strong> 2022 - 2023
  </p>

  <Image src="/images/projects/full_domain.png" alt="logo" width="400" height="200" />
  <br />
  <hr />
  <br />
  {/* Section 1: Resources */}
  <h4 style={{marginTop: 0, marginRight: 0}}>Resources</h4>
  <ul style={{ listStyle: 'none', paddingLeft: '0' }}>
    <li><a href="https://github.com/illiumst/marl-factory-grid/" target="_blank" rel="noopener noreferrer"><i className="fab fa-fw fa-github" aria-hidden="true"></i> GitHub Repo</a></li>
    <li><a href="https://pypi.org/project/Marl-Factory-Grid/" target="_blank" rel="noopener noreferrer"><i className="fab fa-fw fa-python" aria-hidden="true"></i> Install via PyPI</a></li>
    <li><a href="https://marl-factory-grid.readthedocs.io/en/latest/" target="_blank" rel="noopener noreferrer"><i className="fas fa-fw fa-book" aria-hidden="true"></i> ReadTheDocs</a></li>
    <li><i className="fas fa-fw fa-file-alt" aria-hidden="true"></i> <Cite bibtexKey="altmann2024emergence" /></li>
  </ul>
</InfoBox>

To facilitate research into these phenomena, key contributions included the development of specialized simulation tools:

**1. High-Performance MARL Simulation Environment:**
*   A flexible and efficient simulation environment was developed in Python, adhering to the [Gymnasium (formerly Gym) API specification](https://gymnasium.farama.org/main/).
*   **Purpose:** Designed specifically for training and evaluating reinforcement learning algorithms in multi-agent contexts prone to emergent behaviors.
*   **Features:**
    *   **Modularity:** Supports diverse scenarios through configurable `modules` and `configurations`.
    *   **Observation/Action Spaces:** Handles complex agent interactions, including per-agent observations and sequential/multi-agent action coordination.
    *   **Performance:** Optimized for efficient simulation runs, enabling extensive experimentation.

**2. Unity-Based Demonstrator Unit:**
*   A complementary visualization tool was created using the Unity engine.
*   **Purpose:** Allows for the replay, inspection, and detailed analysis of specific simulation scenarios and agent interactions.
*   **Utility:** Aids researchers in identifying and understanding the mechanisms behind observed emergent dynamics.
*   [View Demonstrator on GitHub](https://github.com/illiumst/F-IKS_demonstrator)

<div className="text-center">
  <Image src="/images/projects/rel_emergence.png" alt="Diagram illustrating the concept of emergence from interactions between agents and environment" width="800" height="300"/>
  <figcaption>Conceptual relationship defining emergence in multi-agent systems.</figcaption>
</div>

This project involved close collaboration with industry-focused researchers, software development adhering to modern standards, and deep investigation into the theoretical underpinnings of emergence and safety in MARL systems. The developed tools provide a valuable platform for continued research in this critical area.