mirror of
https://github.com/illiumst/marl-factory-grid.git
synced 2025-05-22 14:56:43 +02:00
readme improvement
This commit is contained in:
Chanumask
parent
f62afefa20
commit
e1b94c4bb0
78
README.md
78
README.md
@ -2,16 +2,27 @@
|
||||
|
||||
## Tackling emergent dysfunctions (EDYs) in cooperation with Fraunhofer-IKS.
|
||||
|
||||
Collaborating with Fraunhofer-IKS, this project is dedicated to investigating Emergent Dysfunctions (EDYs)
|
||||
within multi-agent environments.
|
||||
Collaborating with Fraunhofer-IKS, this project is dedicated to investigating Emergent Dysfunctions (EDYs) within
|
||||
multi-agent environments. In multi-agent reinforcement learning (MARL), a population of agents learns by interacting
|
||||
with each other in a shared environment and adapt their behavior based on the feedback they receive from the environment
|
||||
and the actions of other agents.
|
||||
|
||||
In this context, emergent behavior describes spontaneous behaviors resulting from interactions among agents and
|
||||
environmental stimuli, rather than explicit programming. This promotes natural, adaptable behavior, increases system
|
||||
unpredictability for dynamic learning , enables diverse strategies, and encourages collective intelligence for complex
|
||||
problem-solving. However, the complex dynamics of the environment also give rise to emerging dysfunctions—unexpected
|
||||
issues from agent interactions. This research aims to enhance our understanding of EDYs and their impact on multi-agent
|
||||
systems.
|
||||
|
||||
### Project Objectives:
|
||||
|
||||
- Create an environment that provokes emerging dysfunctions.
|
||||
|
||||
- This is achieved by creating a high level of background noise in the domain, where various entities perform diverse tasks,
|
||||
- This is achieved by creating a high level of background noise in the domain, where various entities perform
|
||||
diverse tasks,
|
||||
resulting in a deliberately chaotic dynamic.
|
||||
- The goal is to observe and analyze naturally occurring emergent dysfunctions within the complexity generated in this dynamic environment.
|
||||
- The goal is to observe and analyze naturally occurring emergent dysfunctions within the complexity generated in
|
||||
this dynamic environment.
|
||||
|
||||
|
||||
- Observational Framework:
|
||||
@ -28,19 +39,24 @@ within multi-agent environments.
|
||||
|
||||
- Placeholders
|
||||
|
||||
- One can provide an agent with a placeholder observation that contains no information and offers no meaningful insights.
|
||||
- Later, when the environment expands and introduces additional entities available for observation, these new observations can be provided to the agent.
|
||||
- This allows for processes such as retraining on an already initialized policy and fine-tuning to enhance the agent's performance based on the enriched information.
|
||||
|
||||
- One can provide an agent with a placeholder observation that contains no information and offers no meaningful
|
||||
insights.
|
||||
- Later, when the environment expands and introduces additional entities available for observation, these new
|
||||
observations can be provided to the agent.
|
||||
- This allows for processes such as retraining on an already initialized policy and fine-tuning to enhance the
|
||||
agent's performance based on the enriched information.
|
||||
|
||||
## Setup
|
||||
Install this environment using `pip install marl-factory-grid`. For more information refer to ['installation'](docs/source/installation.rst).
|
||||
|
||||
Install this environment using `pip install marl-factory-grid`. For more information refer
|
||||
to ['installation'](docs/source/installation.rst).
|
||||
Refer to [quickstart](_quickstart) for specific scenarios.
|
||||
|
||||
## Usage
|
||||
|
||||
The majority of environment objects, including entities, rules, and assets, can be loaded automatically.
|
||||
Simply specify the requirements of your environment in a [*yaml*-config file](marl_factory_grid/configs/default_config.yaml).
|
||||
Simply specify the requirements of your environment in a [
|
||||
*yaml*-config file](marl_factory_grid/configs/default_config.yaml).
|
||||
|
||||
If you only plan on using the environment without making any modifications, use ``quickstart_use``.
|
||||
This creates a default config-file and another one that lists all possible options of the environment.
|
||||
@ -48,49 +64,65 @@ Also, it generates an initial script where an agent is executed in the specified
|
||||
For further details on utilizing the environment, refer to ['usage'](docs/source/usage.rst).
|
||||
|
||||
Existing modules include a variety of functionalities within the environment:
|
||||
- [Agents](marl_factory_grid/algorithms) implement either static strategies or learning algorithms based on the specific configuration.
|
||||
|
||||
- [Agents](marl_factory_grid/algorithms) implement either static strategies or learning algorithms based on the specific
|
||||
configuration.
|
||||
- Their action set includes opening [door entities](marl_factory_grid/modules/doors/entitites.py), cleaning
|
||||
[dirt](marl_factory_grid/modules/clean_up/entitites.py), picking up [items](marl_factory_grid/modules/items/entitites.py) and
|
||||
delivering them to designated drop-off locations.
|
||||
- Agents are equipped with a [battery](marl_factory_grid/modules/batteries/entitites.py) that gradually depletes over time if not charged at a chargepod.
|
||||
- The [maintainer](marl_factory_grid/modules/maintenance/entities.py) aims to repair [machines](marl_factory_grid/modules/machines/entitites.py) that lose health over time.
|
||||
[dirt](marl_factory_grid/modules/clean_up/entitites.py), picking
|
||||
up [items](marl_factory_grid/modules/items/entitites.py) and
|
||||
delivering them to designated drop-off locations.
|
||||
- Agents are equipped with a [battery](marl_factory_grid/modules/batteries/entitites.py) that gradually depletes over
|
||||
time if not charged at a chargepod.
|
||||
- The [maintainer](marl_factory_grid/modules/maintenance/entities.py) aims to
|
||||
repair [machines](marl_factory_grid/modules/machines/entitites.py) that lose health over time.
|
||||
|
||||
## Customization
|
||||
|
||||
If you plan on modifying the environment by for example adding entities or rules, use ``quickstart_modify``.
|
||||
This creates a template module and a script that runs an agent, incorporating the generated module.
|
||||
More information on how to modify the levels, entities, groups, rules and assets goto [modifications](docs/source/modifications.rst).
|
||||
More information on how to modify the levels, entities, groups, rules and assets
|
||||
goto [modifications](docs/source/modifications.rst).
|
||||
|
||||
### Levels
|
||||
Varying levels are created by defining Walls, Floor or Doors in *.txt*-files (see [levels](marl_factory_grid/levels) for examples).
|
||||
|
||||
Varying levels are created by defining Walls, Floor or Doors in *.txt*-files (see [levels](marl_factory_grid/levels) for
|
||||
examples).
|
||||
Define which *level* to use in your *configfile* as:
|
||||
|
||||
```yaml
|
||||
General:
|
||||
level_name: rooms # 'double', 'large', 'simple', ...
|
||||
```
|
||||
|
||||
... or create your own , maybe with the help of [asciiflow.com](https://asciiflow.com/#/).
|
||||
Make sure to use `#` as [Walls](marl_factory_grid/environment/entity/wall.py), `-` as free (walkable) floor, `D` for [Doors](./modules/doors/entities.py).
|
||||
Make sure to use `#` as [Walls](marl_factory_grid/environment/entity/wall.py), `-` as free (walkable) floor, `D`
|
||||
for [Doors](./modules/doors/entities.py).
|
||||
Other Entites (define you own) may bring their own `Symbols`
|
||||
|
||||
### Entites
|
||||
|
||||
Entites are [Objects](marl_factory_grid/environment/entity/object.py) that can additionally be assigned a position.
|
||||
Abstract Entities are provided.
|
||||
|
||||
### Groups
|
||||
[Groups](marl_factory_grid/environment/groups/objects.py) are entity Sets that provide administrative access to all group members.
|
||||
|
||||
[Groups](marl_factory_grid/environment/groups/objects.py) are entity Sets that provide administrative access to all
|
||||
group members.
|
||||
All [Entites](marl_factory_grid/environment/entity/global_entities.py) are available at runtime as EnvState property.
|
||||
|
||||
|
||||
### Rules
|
||||
|
||||
[Rules](marl_factory_grid/environment/entity/object.py) define how the environment behaves on microscale.
|
||||
Each of the hookes (`on_init`, `pre_step`, `on_step`, '`post_step`', `on_done`)
|
||||
provide env-access to implement customn logic, calculate rewards, or gather information.
|
||||
provide env-access to implement custom logic, calculate rewards, or gather information.
|
||||
|
||||
|
||||
|
||||
[Results](marl_factory_grid/environment/entity/object.py) provide a way to return `rule` evaluations such as rewards and state reports
|
||||
back to the environment.
|
||||
[Results](marl_factory_grid/environment/entity/object.py) provide a way to return `rule` evaluations such as rewards and
|
||||
state reports back to the environment.
|
||||
|
||||
### Assets
|
||||
|
||||
Make sure to bring your own assets for each Entity living in the Gridworld as the `Renderer` relies on it.
|
||||
PNG-files (transparent background) of square aspect-ratio should do the job, in general.
|
||||
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user