diff --git a/README.md b/README.md index af256a2..b93f9eb 100644 --- a/README.md +++ b/README.md @@ -1,46 +1,62 @@ # About EDYS -## Tackling emergent dysfunctions (EDYs) in cooperation with Fraunhofer-IKS. +## Tackling emergent dysfunctions (EDYs) in cooperation with Fraunhofer-IKS. -Collaborating with Fraunhofer-IKS, this project is dedicated to investigating Emergent Dysfunctions (EDYs) -within multi-agent environments. +Collaborating with Fraunhofer-IKS, this project is dedicated to investigating Emergent Dysfunctions (EDYs) within +multi-agent environments. In multi-agent reinforcement learning (MARL), a population of agents learns by interacting +with each other in a shared environment and adapt their behavior based on the feedback they receive from the environment +and the actions of other agents. + +In this context, emergent behavior describes spontaneous behaviors resulting from interactions among agents and +environmental stimuli, rather than explicit programming. This promotes natural, adaptable behavior, increases system +unpredictability for dynamic learning , enables diverse strategies, and encourages collective intelligence for complex +problem-solving. However, the complex dynamics of the environment also give rise to emerging dysfunctions—unexpected +issues from agent interactions. This research aims to enhance our understanding of EDYs and their impact on multi-agent +systems. ### Project Objectives: - Create an environment that provokes emerging dysfunctions. - - This is achieved by creating a high level of background noise in the domain, where various entities perform diverse tasks, - resulting in a deliberately chaotic dynamic. - - The goal is to observe and analyze naturally occurring emergent dysfunctions within the complexity generated in this dynamic environment. + - This is achieved by creating a high level of background noise in the domain, where various entities perform + diverse tasks, + resulting in a deliberately chaotic dynamic. + - The goal is to observe and analyze naturally occurring emergent dysfunctions within the complexity generated in + this dynamic environment. - Observational Framework: - - The project introduces an environment that is designed to capture dysfunctions as they naturally occur. - - The environment allows for continuous monitoring of agent behaviors, actions, and interactions. - - Tracking emergent dysfunctions in real-time provides valuable data for analysis and understanding. + - The project introduces an environment that is designed to capture dysfunctions as they naturally occur. + - The environment allows for continuous monitoring of agent behaviors, actions, and interactions. + - Tracking emergent dysfunctions in real-time provides valuable data for analysis and understanding. - Compatibility - - The Framework allows learning entities from different manufacturers and projects with varying representations - of actions and observations to interact seamlessly within the environment. + - The Framework allows learning entities from different manufacturers and projects with varying representations + of actions and observations to interact seamlessly within the environment. - Placeholders - - - One can provide an agent with a placeholder observation that contains no information and offers no meaningful insights. - - Later, when the environment expands and introduces additional entities available for observation, these new observations can be provided to the agent. - - This allows for processes such as retraining on an already initialized policy and fine-tuning to enhance the agent's performance based on the enriched information. + - One can provide an agent with a placeholder observation that contains no information and offers no meaningful + insights. + - Later, when the environment expands and introduces additional entities available for observation, these new + observations can be provided to the agent. + - This allows for processes such as retraining on an already initialized policy and fine-tuning to enhance the + agent's performance based on the enriched information. ## Setup -Install this environment using `pip install marl-factory-grid`. For more information refer to ['installation'](docs/source/installation.rst). + +Install this environment using `pip install marl-factory-grid`. For more information refer +to ['installation'](docs/source/installation.rst). Refer to [quickstart](_quickstart) for specific scenarios. ## Usage -The majority of environment objects, including entities, rules, and assets, can be loaded automatically. -Simply specify the requirements of your environment in a [*yaml*-config file](marl_factory_grid/configs/default_config.yaml). +The majority of environment objects, including entities, rules, and assets, can be loaded automatically. +Simply specify the requirements of your environment in a [ +*yaml*-config file](marl_factory_grid/configs/default_config.yaml). If you only plan on using the environment without making any modifications, use ``quickstart_use``. This creates a default config-file and another one that lists all possible options of the environment. @@ -48,49 +64,65 @@ Also, it generates an initial script where an agent is executed in the specified For further details on utilizing the environment, refer to ['usage'](docs/source/usage.rst). Existing modules include a variety of functionalities within the environment: -- [Agents](marl_factory_grid/algorithms) implement either static strategies or learning algorithms based on the specific configuration. + +- [Agents](marl_factory_grid/algorithms) implement either static strategies or learning algorithms based on the specific + configuration. - Their action set includes opening [door entities](marl_factory_grid/modules/doors/entitites.py), cleaning -[dirt](marl_factory_grid/modules/clean_up/entitites.py), picking up [items](marl_factory_grid/modules/items/entitites.py) and -delivering them to designated drop-off locations. -- Agents are equipped with a [battery](marl_factory_grid/modules/batteries/entitites.py) that gradually depletes over time if not charged at a chargepod. -- The [maintainer](marl_factory_grid/modules/maintenance/entities.py) aims to repair [machines](marl_factory_grid/modules/machines/entitites.py) that lose health over time. + [dirt](marl_factory_grid/modules/clean_up/entitites.py), picking + up [items](marl_factory_grid/modules/items/entitites.py) and + delivering them to designated drop-off locations. +- Agents are equipped with a [battery](marl_factory_grid/modules/batteries/entitites.py) that gradually depletes over + time if not charged at a chargepod. +- The [maintainer](marl_factory_grid/modules/maintenance/entities.py) aims to + repair [machines](marl_factory_grid/modules/machines/entitites.py) that lose health over time. ## Customization If you plan on modifying the environment by for example adding entities or rules, use ``quickstart_modify``. -This creates a template module and a script that runs an agent, incorporating the generated module. -More information on how to modify the levels, entities, groups, rules and assets goto [modifications](docs/source/modifications.rst). +This creates a template module and a script that runs an agent, incorporating the generated module. +More information on how to modify the levels, entities, groups, rules and assets +goto [modifications](docs/source/modifications.rst). ### Levels -Varying levels are created by defining Walls, Floor or Doors in *.txt*-files (see [levels](marl_factory_grid/levels) for examples). -Define which *level* to use in your *configfile* as: + +Varying levels are created by defining Walls, Floor or Doors in *.txt*-files (see [levels](marl_factory_grid/levels) for +examples). +Define which *level* to use in your *configfile* as: + ```yaml General: - level_name: rooms # 'double', 'large', 'simple', ... + level_name: rooms # 'double', 'large', 'simple', ... ``` + ... or create your own , maybe with the help of [asciiflow.com](https://asciiflow.com/#/). -Make sure to use `#` as [Walls](marl_factory_grid/environment/entity/wall.py), `-` as free (walkable) floor, `D` for [Doors](./modules/doors/entities.py). +Make sure to use `#` as [Walls](marl_factory_grid/environment/entity/wall.py), `-` as free (walkable) floor, `D` +for [Doors](./modules/doors/entities.py). Other Entites (define you own) may bring their own `Symbols` ### Entites + Entites are [Objects](marl_factory_grid/environment/entity/object.py) that can additionally be assigned a position. Abstract Entities are provided. ### Groups -[Groups](marl_factory_grid/environment/groups/objects.py) are entity Sets that provide administrative access to all group members. + +[Groups](marl_factory_grid/environment/groups/objects.py) are entity Sets that provide administrative access to all +group members. All [Entites](marl_factory_grid/environment/entity/global_entities.py) are available at runtime as EnvState property. - ### Rules + [Rules](marl_factory_grid/environment/entity/object.py) define how the environment behaves on microscale. -Each of the hookes (`on_init`, `pre_step`, `on_step`, '`post_step`', `on_done`) -provide env-access to implement customn logic, calculate rewards, or gather information. +Each of the hookes (`on_init`, `pre_step`, `on_step`, '`post_step`', `on_done`) +provide env-access to implement custom logic, calculate rewards, or gather information. ![Hooks](../../images/Hooks_FIKS.png) -[Results](marl_factory_grid/environment/entity/object.py) provide a way to return `rule` evaluations such as rewards and state reports -back to the environment. +[Results](marl_factory_grid/environment/entity/object.py) provide a way to return `rule` evaluations such as rewards and +state reports back to the environment. + ### Assets + Make sure to bring your own assets for each Entity living in the Gridworld as the `Renderer` relies on it. PNG-files (transparent background) of square aspect-ratio should do the job, in general.