steffen/marl-factory-grid
1
0
Fork 0
mirror of https://github.com/illiumst/marl-factory-grid.git synced 2025-05-22 14:56:43 +02:00
Code Issues Packages Projects Releases Wiki Activity

readme improvement

Browse Source
This commit is contained in:
Chanumask 2024-02-01 14:02:11 +01:00
parent f62afefa20
commit e1b94c4bb0
1 changed files with 67 additions and 35 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

102
README.md
View File

@ -1,46 +1,62 @@
# About EDYS # About EDYS
## Tackling emergent dysfunctions (EDYs) in cooperation with Fraunhofer-IKS. ## Tackling emergent dysfunctions (EDYs) in cooperation with Fraunhofer-IKS.
Collaborating with Fraunhofer-IKS, this project is dedicated to investigating Emergent Dysfunctions (EDYs) Collaborating with Fraunhofer-IKS, this project is dedicated to investigating Emergent Dysfunctions (EDYs) within
within multi-agent environments. multi-agent environments. In multi-agent reinforcement learning (MARL), a population of agents learns by interacting
with each other in a shared environment and adapt their behavior based on the feedback they receive from the environment
and the actions of other agents.
In this context, emergent behavior describes spontaneous behaviors resulting from interactions among agents and
environmental stimuli, rather than explicit programming. This promotes natural, adaptable behavior, increases system
unpredictability for dynamic learning , enables diverse strategies, and encourages collective intelligence for complex
problem-solving. However, the complex dynamics of the environment also give rise to emerging dysfunctions—unexpected
issues from agent interactions. This research aims to enhance our understanding of EDYs and their impact on multi-agent
systems.
### Project Objectives: ### Project Objectives:
- Create an environment that provokes emerging dysfunctions. - Create an environment that provokes emerging dysfunctions.
- This is achieved by creating a high level of background noise in the domain, where various entities perform diverse tasks, - This is achieved by creating a high level of background noise in the domain, where various entities perform
resulting in a deliberately chaotic dynamic. diverse tasks,
- The goal is to observe and analyze naturally occurring emergent dysfunctions within the complexity generated in this dynamic environment. resulting in a deliberately chaotic dynamic.
- The goal is to observe and analyze naturally occurring emergent dysfunctions within the complexity generated in
this dynamic environment.
- Observational Framework: - Observational Framework:
- The project introduces an environment that is designed to capture dysfunctions as they naturally occur. - The project introduces an environment that is designed to capture dysfunctions as they naturally occur.
- The environment allows for continuous monitoring of agent behaviors, actions, and interactions. - The environment allows for continuous monitoring of agent behaviors, actions, and interactions.
- Tracking emergent dysfunctions in real-time provides valuable data for analysis and understanding. - Tracking emergent dysfunctions in real-time provides valuable data for analysis and understanding.
- Compatibility - Compatibility
- The Framework allows learning entities from different manufacturers and projects with varying representations - The Framework allows learning entities from different manufacturers and projects with varying representations
of actions and observations to interact seamlessly within the environment. of actions and observations to interact seamlessly within the environment.
- Placeholders - Placeholders
- One can provide an agent with a placeholder observation that contains no information and offers no meaningful insights.
- Later, when the environment expands and introduces additional entities available for observation, these new observations can be provided to the agent.
- This allows for processes such as retraining on an already initialized policy and fine-tuning to enhance the agent's performance based on the enriched information.
- One can provide an agent with a placeholder observation that contains no information and offers no meaningful
insights.
- Later, when the environment expands and introduces additional entities available for observation, these new
observations can be provided to the agent.
- This allows for processes such as retraining on an already initialized policy and fine-tuning to enhance the
agent's performance based on the enriched information.
## Setup ## Setup
Install this environment using `pip install marl-factory-grid`. For more information refer to ['installation'](docs/source/installation.rst).
Install this environment using `pip install marl-factory-grid`. For more information refer
to ['installation'](docs/source/installation.rst).
Refer to [quickstart](_quickstart) for specific scenarios. Refer to [quickstart](_quickstart) for specific scenarios.
## Usage ## Usage
The majority of environment objects, including entities, rules, and assets, can be loaded automatically. The majority of environment objects, including entities, rules, and assets, can be loaded automatically.
Simply specify the requirements of your environment in a [*yaml*-config file](marl_factory_grid/configs/default_config.yaml). Simply specify the requirements of your environment in a [
*yaml*-config file](marl_factory_grid/configs/default_config.yaml).
If you only plan on using the environment without making any modifications, use ``quickstart_use``. If you only plan on using the environment without making any modifications, use ``quickstart_use``.
This creates a default config-file and another one that lists all possible options of the environment. This creates a default config-file and another one that lists all possible options of the environment.
@ -48,49 +64,65 @@ Also, it generates an initial script where an agent is executed in the specified
For further details on utilizing the environment, refer to ['usage'](docs/source/usage.rst). For further details on utilizing the environment, refer to ['usage'](docs/source/usage.rst).
Existing modules include a variety of functionalities within the environment: Existing modules include a variety of functionalities within the environment:
- [Agents](marl_factory_grid/algorithms) implement either static strategies or learning algorithms based on the specific configuration.
- [Agents](marl_factory_grid/algorithms) implement either static strategies or learning algorithms based on the specific
configuration.
- Their action set includes opening [door entities](marl_factory_grid/modules/doors/entitites.py), cleaning - Their action set includes opening [door entities](marl_factory_grid/modules/doors/entitites.py), cleaning
[dirt](marl_factory_grid/modules/clean_up/entitites.py), picking up [items](marl_factory_grid/modules/items/entitites.py) and [dirt](marl_factory_grid/modules/clean_up/entitites.py), picking
delivering them to designated drop-off locations. up [items](marl_factory_grid/modules/items/entitites.py) and
- Agents are equipped with a [battery](marl_factory_grid/modules/batteries/entitites.py) that gradually depletes over time if not charged at a chargepod. delivering them to designated drop-off locations.
- The [maintainer](marl_factory_grid/modules/maintenance/entities.py) aims to repair [machines](marl_factory_grid/modules/machines/entitites.py) that lose health over time. - Agents are equipped with a [battery](marl_factory_grid/modules/batteries/entitites.py) that gradually depletes over
time if not charged at a chargepod.
- The [maintainer](marl_factory_grid/modules/maintenance/entities.py) aims to
repair [machines](marl_factory_grid/modules/machines/entitites.py) that lose health over time.
## Customization ## Customization
If you plan on modifying the environment by for example adding entities or rules, use ``quickstart_modify``. If you plan on modifying the environment by for example adding entities or rules, use ``quickstart_modify``.
This creates a template module and a script that runs an agent, incorporating the generated module. This creates a template module and a script that runs an agent, incorporating the generated module.
More information on how to modify the levels, entities, groups, rules and assets goto [modifications](docs/source/modifications.rst). More information on how to modify the levels, entities, groups, rules and assets
goto [modifications](docs/source/modifications.rst).
### Levels ### Levels
Varying levels are created by defining Walls, Floor or Doors in *.txt*-files (see [levels](marl_factory_grid/levels) for examples).
Define which *level* to use in your *configfile* as: Varying levels are created by defining Walls, Floor or Doors in *.txt*-files (see [levels](marl_factory_grid/levels) for
examples).
Define which *level* to use in your *configfile* as:
```yaml ```yaml
General: General:
level_name: rooms # 'double', 'large', 'simple', ... level_name: rooms # 'double', 'large', 'simple', ...
``` ```
... or create your own , maybe with the help of [asciiflow.com](https://asciiflow.com/#/). ... or create your own , maybe with the help of [asciiflow.com](https://asciiflow.com/#/).
Make sure to use `#` as [Walls](marl_factory_grid/environment/entity/wall.py), `-` as free (walkable) floor, `D` for [Doors](./modules/doors/entities.py). Make sure to use `#` as [Walls](marl_factory_grid/environment/entity/wall.py), `-` as free (walkable) floor, `D`
for [Doors](./modules/doors/entities.py).
Other Entites (define you own) may bring their own `Symbols` Other Entites (define you own) may bring their own `Symbols`
### Entites ### Entites
Entites are [Objects](marl_factory_grid/environment/entity/object.py) that can additionally be assigned a position. Entites are [Objects](marl_factory_grid/environment/entity/object.py) that can additionally be assigned a position.
Abstract Entities are provided. Abstract Entities are provided.
### Groups ### Groups
[Groups](marl_factory_grid/environment/groups/objects.py) are entity Sets that provide administrative access to all group members.
[Groups](marl_factory_grid/environment/groups/objects.py) are entity Sets that provide administrative access to all
group members.
All [Entites](marl_factory_grid/environment/entity/global_entities.py) are available at runtime as EnvState property. All [Entites](marl_factory_grid/environment/entity/global_entities.py) are available at runtime as EnvState property.
### Rules ### Rules
[Rules](marl_factory_grid/environment/entity/object.py) define how the environment behaves on microscale. [Rules](marl_factory_grid/environment/entity/object.py) define how the environment behaves on microscale.
Each of the hookes (`on_init`, `pre_step`, `on_step`, '`post_step`', `on_done`) Each of the hookes (`on_init`, `pre_step`, `on_step`, '`post_step`', `on_done`)
provide env-access to implement customn logic, calculate rewards, or gather information. provide env-access to implement custom logic, calculate rewards, or gather information.
![Hooks](../../images/Hooks_FIKS.png) ![Hooks](../../images/Hooks_FIKS.png)
[Results](marl_factory_grid/environment/entity/object.py) provide a way to return `rule` evaluations such as rewards and state reports [Results](marl_factory_grid/environment/entity/object.py) provide a way to return `rule` evaluations such as rewards and
back to the environment. state reports back to the environment.
### Assets ### Assets
Make sure to bring your own assets for each Entity living in the Gridworld as the `Renderer` relies on it. Make sure to bring your own assets for each Entity living in the Gridworld as the `Renderer` relies on it.
PNG-files (transparent background) of square aspect-ratio should do the job, in general. PNG-files (transparent background) of square aspect-ratio should do the job, in general.