EAGERx as dependency instead of submodule. (#228)

* EAGERx as dependeny instead of submodule. Docs and demos updated accordingly

* Fix typo

* Minor fixes

* Update test_pep8.py

* Loosen pyqglet requirements to solve dependency issue

* Loosen pyglet requitement to solve dependency issue

* Update dependencies

* Update dependencies

* Update EAGERx

* Fix typo

* Add rendering toggle to eagerx demos

Co-authored-by: Jelle Luijkx <[email protected]>
Co-authored-by: ad-daniel <[email protected]>

Author: jelledouwe

Diff for: .gitmodules

@@ -1,8 +1,3 @@
 [submodule "src/opendr/perception/panoptic_segmentation/efficient_ps/algorithm/EfficientPS"]
 	path = src/opendr/perception/panoptic_segmentation/efficient_ps/algorithm/EfficientPS
 	url = https://github.com/DeepSceneSeg/EfficientPS.git
-[submodule "src/opendr/utils/eagerx/eagerx"]
-	path = projects/control/eagerx/eagerx
-	url = https://github.com/eager-dev/eagerx.git
-	branch = opendr_v1.0
-	

Diff for: Makefile

@@ -39,7 +39,6 @@ install_compilation_dependencies:
 	@+cd dependencies; ./install_onnx.sh
 	@+make --silent -C src/opendr/control/mobile_manipulation $(TARGET) OPENDR_HOME="$(OPENDR_HOME)";
 	@+make --silent -C src/opendr/control/single_demo_grasp $(TARGET) OPENDR_HOME="$(OPENDR_HOME)";
-	@+make --silent -C projects/control/eagerx $(TARGET) OPENDR_HOME="$(OPENDR_HOME)";
 
 styletest:
 	@+echo "Testing file licences and code-style"
@@ -77,4 +76,3 @@ help:
 	@+echo
 	@+echo -e "\033[32;1mNote:\033[0m You seem to have a processor with $(NUMBER_OF_PROCESSORS) virtual cores,"
 	@+echo -e "      hence the \033[33;1m-j$(THREADS)\033[0m option to speed-up the compilation."
-

Diff for: docs/reference/eagerx.md

@@ -20,31 +20,32 @@ The source code for EAGERx is available [here](https://github.com/eager-dev/eage
 Documentation is available online: [https://eagerx.readthedocs.io](https://eagerx.readthedocs.io)
 
 
-### Examples
+### EAGERx Demos
 
 **Prerequisites**: EAGERx requires ROS Noetic and Python 3.8 to be installed.
 
-After installation of the OpenDR toolkit, you can run one of the available examples as follows.
-
-First source the workspace:
-
+1. **[demo_full_state](../../projects/control/eagerx/demos/demo_full_state.py)**:  
+   Here, we wrap the OpenAI gym within EAGERx.
+   The agent learns to map low-dimensional angular observations to torques.
+2. **[demo_pid](../../projects/control/eagerx/demos/demo_pid.py)**:   
+   Here, we add a PID controller, tuned to stabilize the pendulum in the upright position, as a pre-processing node.
+   The agent now maps low-dimensional angular observations to reference torques.
+   In turn, the reference torques are converted to torques by the PID controller, and applied to the system.
+3. **[demo_classifier](../../projects/control/eagerx/demos/demo_classifier.py)**:   
+   Instead of using low-dimensional angular observations, the environment now produces pixel images of the pendulum.
+   In order to speed-up learning, we use a pre-trained classifier to convert these pixel images to estimated angular observations.
+   Then, the agent uses these estimated angular observations similarly as in 'demo_2_pid' to successfully swing-up the pendulum.
+   
+Example usage:
 ```bash
-source $OPENDR_HOME/projects/control/eagerx/eagerx_ws/devel/setup.bash
+cd $OPENDR_HOME/projects/control/eagerx/demos
+python3 [demo_name]
 ```
 
-Now you can run one of the demos in the terminal where you sourced the workspace:
-
-```bash
-source $OPENDR_HOME/projects/control/eagerx/eagerx_ws/devel/setup.bash
-rosrun eagerx_example_opendr [demo_name]
-```
+where possible values for [demo_name] are: *demo_full_state.py*, *demo_pid.py*, *demo_classifier.py*
 
-where possible values for [demo_name] are:
-- **demo_1_full_state**: Here, we wrap the OpenAI gym within EAGERx.
-The agent learns to map low-dimensional angular observations to torques.
-- **demo_2_pid**: Here, we add a PID controller, tuned to stabilize the pendulum in the upright position, as a pre-processing node.
-The agent now maps low-dimensional angular observations to reference torques.
-In turn, the reference torques are converted to torques by the PID controller, and applied to the system.
-- **demo_3_classifier**: Instead of using low-dimensional angular observations, the environment now produces pixel images of the pendulum.
-In order to speed-up learning, we use a pre-trained classifier to convert these pixel images to estimated angular observations.
-Then, the agent uses these estimated angular observations similarly as in 'demo_2_pid' to successfully swing-up the pendulum.
+Setting `--device cpu` performs training and inference on CPU.  
+Setting `--name example` sets the name of the environment.  
+Setting `--eps 200` sets the number of training episodes.  
+Setting `--eval-eps 10` sets the number of evaluation episodes.
+Adding `--render` enables rendering of the environment.

Diff for: projects/control/eagerx/Makefile

@@ -1,66 +1,49 @@
-[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
-
-# EAGERx
+# EAGERx Demos
 
 Engine Agnostic Gym Environment with Reactive extension (EAGERx) is a toolkit that will allow users to apply (deep) reinforcement learning for both simulated and real robots as well as combinations thereof.
 Documentation is available [here](../../../docs/reference/eagerx.md).
 The source code of EAGERx is available [here](https://github.com/eager-dev/eagerx)
 
-### Installation
-
 **Prerequisites**: EAGERx requires ROS Noetic and Python 3.8 to be installed.
 
-Follow the OpenDR installation instructions.
-Next, one should also install the appropriate runtime dependencies:
-
-```bash
-cd $OPENDR_HOME
-make install_runtime_dependencies
-```
-
-Now the user is ready to go!
-
-
-### Examples
-
-**Prerequisites**: EAGERx requires ROS Noetic and Python 3.8 to be installed.
-
-After installation of the OpenDR toolkit, you can run one of the available examples as follows.
-
-First source the workspace:
-
-```bash
-source $OPENDR_HOME/projects/control/eagerx/eagerx_ws/devel/setup.bash
-```
-
-Now you can run one of the demos in the terminal where you sourced the workspace:
-
+This folder contains minimal code usage examples that showcase some of EAGERx's features. 
+Specifically the following examples are provided:
+1. **[demo_full_state](demos/demo_full_state.py)**:  
+   Here, we wrap the OpenAI gym within EAGERx.
+   The agent learns to map low-dimensional angular observations to torques.
+2. **[demo_pid](demos/demo_pid.py)**:   
+   Here, we add a PID controller, tuned to stabilize the pendulum in the upright position, as a pre-processing node.
+   The agent now maps low-dimensional angular observations to reference torques.
+   In turn, the reference torques are converted to torques by the PID controller, and applied to the system.
+3. **[demo_classifier](demos/demo_classifier.py)**:   
+   Instead of using low-dimensional angular observations, the environment now produces pixel images of the pendulum.
+   In order to speed-up learning, we use a pre-trained classifier to convert these pixel images to estimated angular observations.
+   Then, the agent uses these estimated angular observations similarly as in 'demo_2_pid' to successfully swing-up the pendulum.
+   
+Example usage:
 ```bash
-source $OPENDR_HOME/projects/control/eagerx/eagerx_ws/devel/setup.bash
-rosrun eagerx_example_opendr [demo_name]
+cd $OPENDR_HOME/projects/control/eagerx/demos
+python3 [demo_name]
 ```
 
-where possible values for [demo_name] are:
-- **demo_1_full_state**: Here, we wrap the OpenAI gym within EAGERx.
-The agent learns to map low-dimensional angular observations to torques.
-- **demo_2_pid**: Here, we add a PID controller, tuned to stabilize the pendulum in the upright position, as a pre-processing node.
-The agent now maps low-dimensional angular observations to reference torques.
-In turn, the reference torques are converted to torques by the PID controller, and applied to the system.
-- **demo_3_classifier**: Instead of using low-dimensional angular observations, the environment now produces pixel images of the pendulum.
-In order to speed-up learning, we use a pre-trained classifier to convert these pixel images to estimated angular observations.
-Then, the agent uses these estimated angular observations similarly as in 'demo_2_pid' to successfully swing-up the pendulum.
+where possible values for [demo_name] are: *demo_full_state.py*, *demo_pid.py*, *demo_classifier.py*
 
+Setting `--device cpu` performs training and inference on CPU.  
+Setting `--name example` sets the name of the environment.  
+Setting `--eps 200` sets the number of training episodes.  
+Setting `--eval-eps 10` sets the number of evaluation episodes.
+Adding `--render` enables rendering of the environment.
 
 ## Citing EAGERx
 
 To cite EAGERx in publications:
 ```bibtex
-@misc{eagerx,
-  author = {Van der Heijden, Bas and Luijkx, Jelle},
-  title = {EAGERx: Engine Agnostic Gym Environment with Reactive extension},
-  year = {2021},
-  publisher = {GitHub},
-  journal = {GitHub repository},
-  howpublished = {\url{https://github.com/eager-dev/eagerx}},
+@article{eagerx,
+    author  = {van der Heijden, Bas and Luijkx, Jelle, and Ferranti, Laura and Kober, Jens and Babuska, Robert},
+    title = {EAGER: Engine Agnostic Gym Environment for Robotics},
+    year = {2022},
+    publisher = {GitHub},
+    journal = {GitHub repository},
+    howpublished = {\url{https://github.com/eager-dev/eagerx}}
 }
 ```

Diff for: projects/control/eagerx/README.md

@@ -0,0 +1,94 @@
+#!/usr/bin/env python
+# Copyright 2020-2022 OpenDR European Project
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+
+# EAGERx imports
+from eagerx import Object, Bridge, Node, initialize, log
+from eagerx.core.graph import Graph
+import eagerx.bridges.openai_gym as eagerx_gym
+import eagerx_examples  # noqa: F401
+
+# Import stable-baselines
+import stable_baselines3 as sb
+
+
+def example_classifier(name, eps, eval_eps, device, render=False):
+    # Start roscore & initialize main thread as node
+    initialize("eagerx", anonymous=True, log_level=log.INFO)
+
+    # Define object
+    pendulum = Object.make(
+        "GymObject",
+        "pendulum",
+        sensors=["image", "observation", "reward", "done"],
+        gym_env_id="Pendulum-v0",
+        gym_rate=20,
+        gym_always_render=True,
+        render_shape=[28, 28],
+    )
+
+    # Define PID controller & classifier
+    classifier = Node.make("Classifier", "classifier", rate=20, cam_rate=20, data="../data/with_actions.h5")
+    pid = Node.make("PidController", "pid", rate=20, gains=[8, 1, 0], y_range=[-4, 4])
+
+    # Define graph (agnostic) & connect nodes
+    graph = Graph.create(nodes=[classifier, pid], objects=[pendulum])
+    graph.connect(source=pendulum.sensors.reward, observation="reward")
+    graph.connect(source=pendulum.sensors.done, observation="done")
+    graph.connect(source=classifier.outputs.state, observation="state")
+    # Connect Classifier
+    graph.connect(source=classifier.outputs.state, target=pid.inputs.y)
+    graph.connect(source=pendulum.sensors.image, target=classifier.inputs.image)
+    # Connect PID
+    graph.connect(action="yref", target=pid.inputs.yref)
+    graph.connect(source=pid.outputs.u, target=pendulum.actuators.action)
+    # Add rendering
+    if render:
+        graph.render(source=pendulum.sensors.image, rate=10, display=True)
+
+    # Define bridge
+    bridge = Bridge.make("GymBridge", rate=20)
+
+    # Initialize Environment (agnostic graph +  bridge)
+    env = eagerx_gym.EagerGym(name=name, rate=20, graph=graph, bridge=bridge)
+    if render:
+        env.render(mode='human')
+
+    # Initialize and train stable-baselines model
+    model = sb.SAC("MlpPolicy", env, verbose=1, device=device)
+    model.learn(total_timesteps=int(eps * 200))
+
+    # Evaluate trained policy
+    for i in range(eval_eps):
+        obs, done = env.reset(), False
+        while not done:
+            action, _ = model.predict(obs, deterministic=True)
+            action = env.action_space.sample()
+            obs, reward, done, info = env.step(action)
+    env.shutdown()
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--device", help="Device to use (cpu, cuda)", type=str, default="cpu", choices=["cuda", "cpu"])
+    parser.add_argument("--name", help="Name of the environment", type=str, default="example")
+    parser.add_argument("--eps", help="Number of training episodes", type=int, default=200)
+    parser.add_argument("--eval_eps", help="Number of evaluation episodes", type=int, default=20)
+    parser.add_argument("--render", help="Toggle rendering", action='store_true')
+
+    args = parser.parse_args()
+
+    example_classifier(name=args.name, eps=args.eps, eval_eps=args.eval_eps, device=args.device, render=args.render)

Diff for: projects/control/eagerx/data/with_actions.h5

@@ -0,0 +1,78 @@
+#!/usr/bin/env python
+# Copyright 2020-2022 OpenDR European Project
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+
+# EAGERx imports
+from eagerx import Object, Bridge, initialize, log
+from eagerx.core.graph import Graph
+import eagerx.bridges.openai_gym as eagerx_gym
+import eagerx_examples  # noqa: F401
+
+# Import stable-baselines
+import stable_baselines3 as sb
+
+
+def example_full_state(name, eps, eval_eps, device, render=False):
+    # Start roscore & initialize main thread as node
+    initialize("eagerx", anonymous=True, log_level=log.INFO)
+
+    # Define object
+    sensors = ["observation", "reward", "done"]
+    if render:
+        sensors.append("image")
+    pendulum = Object.make("GymObject", "pendulum", sensors=sensors, gym_env_id="Pendulum-v0", gym_rate=20)
+
+    # Define graph (agnostic) & connect nodes
+    graph = Graph.create(objects=[pendulum])
+    graph.connect(source=pendulum.sensors.observation, observation="observation", window=1)
+    graph.connect(source=pendulum.sensors.reward, observation="reward", window=1)
+    graph.connect(source=pendulum.sensors.done, observation="done", window=1)
+    graph.connect(action="action", target=pendulum.actuators.action, window=1)
+    if render:
+        graph.render(source=pendulum.sensors.image, rate=10, display=False)
+
+    # Define bridge
+    bridge = Bridge.make("GymBridge", rate=20)
+
+    # Initialize Environment (agnostic graph +  bridge)
+    env = eagerx_gym.EagerGym(name=name, rate=20, graph=graph, bridge=bridge)
+    if render:
+        env.render(mode='human')
+
+    # Initialize and train stable-baselines model
+    model = sb.SAC("MlpPolicy", env, verbose=1, device=device)
+    model.learn(total_timesteps=int(eps * 200))
+
+    # Evaluate trained policy
+    for i in range(eval_eps):
+        obs, done = env.reset(), False
+        while not done:
+            action, _ = model.predict(obs, deterministic=True)
+            obs, reward, done, info = env.step(action)
+    env.shutdown()
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--device", help="Device to use (cpu, cuda)", type=str, default="cpu", choices=["cuda", "cpu"])
+    parser.add_argument("--name", help="Name of the environment", type=str, default="example")
+    parser.add_argument("--eps", help="Number of training episodes", type=int, default=200)
+    parser.add_argument("--eval_eps", help="Number of evaluation episodes", type=int, default=20)
+    parser.add_argument("--render", help="Toggle rendering", action='store_true')
+
+    args = parser.parse_args()
+
+    example_full_state(name=args.name, eps=args.eps, eval_eps=args.eval_eps, device=args.device, render=args.render)