68 lines
4.5 KiB
Markdown
68 lines
4.5 KiB
Markdown
---
|
|
layout: single
|
|
title: "AI-Fusion Safety"
|
|
categories: projects
|
|
tags: multi-agent-systems reinforcement-learning safety emergence simulation
|
|
excerpt: "Studied MARL emergence and safety, built simulations with Fraunhofer."
|
|
header:
|
|
teaser: /assets/images/projects/robot.png
|
|
role: Researcher, Software Developer
|
|
skills: Multi-Agent Reinforcement Learning (MARL), Emergence Analysis, AI Safety, Simulation Environment Design, Python, Gymnasium API, Software Engineering, Unity (Visualization), Industry Collaboration
|
|
---
|
|
|
|
<div class="container">
|
|
<div class="sidebar" style="float: right; width: 25%; border: 0.5px grey solid; padding: 15px;">
|
|
<h4 style="margin-top: 0;">Project Resources</h4>
|
|
<ul style="list-style: none; padding-left: 0;">
|
|
<li><a href="https://github.com/illiumst/marl-factory-grid/" target="_blank" rel="noopener noreferrer"><i class="fab fa-fw fa-github" aria-hidden="true"></i> GitHub Repo</a></li>
|
|
<li><a href="https://pypi.org/project/Marl-Factory-Grid/" target="_blank" rel="noopener noreferrer"><i class="fab fa-fw fa-python" aria-hidden="true"></i> Install via PyPI</a></li>
|
|
<li><a href="https://marl-factory-grid.readthedocs.io/en/latest/" target="_blank" rel="noopener noreferrer"><i class="fas fa-fw fa-book" aria-hidden="true"></i> ReadTheDocs</a></li>
|
|
<li><i class="fas fa-fw fa-file-alt" aria-hidden="true"></i> {% cite altmann2024emergence %}</li>
|
|
</ul>
|
|
{: style="margin:0em; padding:0em; width:15em"}
|
|
</div>
|
|
<div class="main-content" style="float: left; width: 75%;">
|
|
|
|
{: .align-left style="padding:0.1em; width:5em"}
|
|
**Project:** AI-Fusion<br>
|
|
**Partner:** [Fraunhofer Institute for Cognitive Systems (IKS)](https://www.iks.fraunhofer.de/)<br>
|
|
**Duration:** 2022 - 2023<br>
|
|
**Objective:** To investigate the detection and mitigation of potentially unsafe emergent behaviors in complex systems composed of multiple interacting AI agents, particularly in scenarios involving heterogeneous agents (e.g., mixed-vendor autonomous systems).
|
|
|
|
In collaboration with Fraunhofer IKS, the AI-Fusion project addressed the critical challenge of understanding and ensuring safety in multi-agent reinforcement learning (MARL) systems. Emergence, defined as the arising of complex, often unpredictable, system-level dynamics from local interactions between agents and their environment, was a central focus due to its implications for system safety and reliability.
|
|
</div>
|
|
</div>
|
|
|
|
---
|
|
|
|
To facilitate research into these phenomena, key contributions included the development of specialized simulation tools:
|
|
|
|
**1. High-Performance MARL Simulation Environment:**
|
|
|
|
* A flexible and efficient simulation environment was developed in Python, adhering to the [Gymnasium (formerly Gym) API specification](https://gymnasium.farama.org/main/).
|
|
* **Purpose:** Designed specifically for training and evaluating reinforcement learning algorithms in multi-agent contexts prone to emergent behaviors.
|
|
* **Features:**
|
|
* **Modularity:** Supports diverse scenarios through configurable `modules` and `configurations`.
|
|
* **Observation/Action Spaces:** Handles complex agent interactions, including per-agent observations and sequential/multi-agent action coordination.
|
|
* **Performance:** Optimized for efficient simulation runs, enabling extensive experimentation.
|
|
|
|
**2. Unity-Based Demonstrator Unit:**
|
|
|
|
* A complementary visualization tool was created using the Unity engine.
|
|
* **Purpose:** Allows for the replay, inspection, and detailed analysis of specific simulation scenarios and agent interactions.
|
|
* **Utility:** Aids researchers in identifying and understanding the mechanisms behind observed emergent dynamics.
|
|
* [View Demonstrator on GitHub](https://github.com/illiumst/F-IKS_demonstrator)
|
|
|
|
<div style="clear: both;"></div>
|
|
|
|
|
|
<center>
|
|
<img src="/assets/images/projects/rel_emergence.png" alt="Diagram illustrating the concept of emergence from interactions between agents and environment" style="padding:0.1em; width:80%">
|
|
<figcaption>Conceptual relationship defining emergence in multi-agent systems.</figcaption>
|
|
</center>
|
|
|
|
|
|
This project involved close collaboration with industry-focused researchers, software development adhering to modern standards, and deep investigation into the theoretical underpinnings of emergence and safety in MARL systems. The developed tools provide a valuable platform for continued research in this critical area.
|
|
|
|
{% include reference.html %}
|