general overhaul, better images, better texts
This commit is contained in:
@ -8,21 +8,21 @@ header:
|
||||
---
|
||||
|
||||
{: .align-left style="padding:0.1em; width:5em"}
|
||||
In cooperation with [Fraunhofer IKS](https://www.iks.fraunhofer.de/) this project aimed to investigate and detect emergent effects in multi-agent reinforcement learning scenarios, i.e., mixed-vendor autonomous systems (AI fusion).
|
||||
Emergence in general refers to emerging dynamics of higher complexity (i.e., sum), which are fed by interacting entities (each other and the environment) of a specific complexity level (regarding their policies and capabilities).
|
||||
{: .align-center style="padding:0.1em; width:30em"}
|
||||
In cooperation with [Fraunhofer IKS](https://www.iks.fraunhofer.de/), this project explored emergent effects in multi-agent reinforcement learning scenarios, such as mixed-vendor autonomous systems. Emergence, defined as complex dynamics arising from interactions among entities and their environment, was a key focus.
|
||||
|
||||
In this context, we developed a full-stack, high-performance environment in Python, following the [gymnasium](https://gymnasium.farama.org/main/) specification for the training of reinforcement learning algorithms.
|
||||
{: .align-center style="padding:0.1em; width:80%"}
|
||||
|
||||
<div class="table-right">
|
||||
|
||||
{: .align-right style="padding:0.5em; width:10em"}
|
||||
|
||||
| [GitHub Repo](https://github.com/illiumst/marl-factory-grid/) | [Read-the-docs](https://marl-factory-grid.readthedocs.io/en/latest/) |
|
||||
| [Install via PyPI](https://pypi.org/project/Marl-Factory-Grid/) | Read the Paper (TBA) |
|
||||
<div class="table-right" style="text-align:right">
|
||||
| {: style="margin:0em; padding:0em; width:15em"} |
|
||||
| [GitHub Repo](https://github.com/illiumst/marl-factory-grid/) |
|
||||
| [Install via PyPI](https://pypi.org/project/Marl-Factory-Grid/) |
|
||||
| [Read-the-docs](https://marl-factory-grid.readthedocs.io/en/latest/) |
|
||||
| Read the Paper (TBA) |
|
||||
|
||||
</div>
|
||||
|
||||
The main differentiation from already established MARL environments is its ability to employ various scenarios as a combination of `modules` and `configurations`. As well as the option to define per-agent observations, including placeholder and combined observation slices (grid-world). Moreover, this environment can handle multi-agent scenarios as well as sequential actions for inter-step observations.
|
||||
We developed a high-performance environment in Python, adhering to the [gymnasium](https://gymnasium.farama.org/main/) specifications, to facilitate reinforcement learning algorithm training.
|
||||
|
||||
Furthermore, we designed and implemented a [Unity demonstrator unit](https://github.com/illiumst/F-IKS_demonstrator) that can load and replay specific pre-recorded scenarios. This way, emerging unwanted and unsafe situations can be replayed and intuitively investigated.
|
||||
This environment uniquely supports a variety of scenarios through `modules` and `configurations`, with capabilities for per-agent observations and handling of multi-agent and sequential actions.
|
||||
|
||||
Additionally, a [Unity demonstrator unit](https://github.com/illiumst/F-IKS_demonstrator) was developed to replay and analyze specific scenarios, aiding in the investigation of emerging dynamics.
|
||||
|
||||
@ -7,19 +7,12 @@ header:
|
||||
teaser: assets/images/projects/arch.png
|
||||
---
|
||||
{: .align-left style="padding:0.1em; width:5em"}
|
||||
During my time at the Mobile Distributed Systems Chair, I also supported my colleagues regarding the setup and maintenance of our workstations, the Windows server hypervisor, our Linux file server, and our networking infrastructure.
|
||||
We orchestrated and utilized multiple systems with varying operating systems, hardware, driver versions, and libraries (damn you CUDA), mostly through Ansible.
|
||||
During my tenure at the Mobile and Distributed Systems Chair, I played a key role in the setup and maintenance of our technical infrastructure, including workstations, Windows server hypervisors, Linux file servers, and networking. Our approach to managing a diverse ecosystem of operating systems, hardware, and libraries involved extensive use of Ansible for orchestration.
|
||||
|
||||
Most recently transferred a major partition of or services to Kubernetes (K3S) while setting up a complete tool chain including Longhorn, Argo CD, Sealed Secrets, and GitLab. For ingress and egress, we set up Traefik as our fully automated proxy manager.
|
||||
This way, we could differentiate between routes through different virtual networks for clients within our networks and external users, i.e., colleagues in the home office or students.
|
||||
Crossing network borders was possible through a self-hosted WireGuard jump-host, which could also be utilized as our full-fledged VPN when traveling aboard.
|
||||
This way, the whole team could access our ZFS-file server from anywhere.
|
||||
I spearheaded the transition of a significant portion of our services to Kubernetes (K3S), implementing a comprehensive toolchain that included Longhorn, Argo CD, Sealed Secrets, and GitLab. For managing ingress and egress, Traefik served as our automated proxy manager, enabling us to efficiently route traffic within our network and accommodate external users securely through WireGuard.
|
||||
|
||||
Starting with self-scripted ML runs on non-reliable SLURM infrastructure (pool PCs could be turned off or used by students, which killed the SLURM agent at any time), I slowly transitioned to runs on individual, high-performance workstations.
|
||||
Lately, those units were automated by weights-and-biases (WandB), which automatically runs experiments in pre-defined Docker containers; for that, we utilized our self-hosted registry through GitLab.
|
||||
My experience extended to optimizing machine learning workflows, transitioning from unreliable SLURM-based setups to automated, high-performance workstation runs using Weights & Biases (WandB) for experiment management, leveraging our self-hosted GitLab registry for Docker container orchestration.
|
||||
|
||||
All in all, I learned a lot about Linux server administration, networking, code-as-infrastructure, and the spirit of the cloud native approach.
|
||||
Nowadays, I thoroughly enjoy the `make your requirements less dumb` approach to things, moving from central services to a bunch of problem-specific (self-hosted) microservices.
|
||||
Moreover, I really like to build systems from the ground up with only the necessary elements on top of a rolling release distribution (Arch-Linux-based).
|
||||
This journey enriched my skills in Linux server administration, networking, infrastructure as code, and cloud-native technologies. It fostered a preference for minimalist, microservice-based architectures, and I've applied these principles to my personal projects, including self-hosting this website and other services, underscoring my commitment to practical, efficient technology solutions.
|
||||
|
||||
I took some concepts into my `home lab`, where I now self-host most of my services, including this website. Isn't that cool? :cool: More of the tech stack I encountered on my journey is listed [here](/about).
|
||||
More of the tech stack I encountered on my journey is listed [here](/about).
|
||||
|
||||
Reference in New Issue
Block a user