Integrating Hugging Face Hub 🤗 (updated) (#5856)

* Add Hugging Face Integration

* Update setup.py

* Update push_to_hf.py

* Update push_to_hf.py

* Update push_to_hf.py

* Update push_to_hf.py

Remove use_auth_token

* Update push_to_hf.py

* Update push_to_hf.py

* Create Huggy

* Update load_from_hf.py

*Change loading to snapshot_download (able to use cache)

* Update push_to_hf.py

* Use create_repo and upload_folder instead of git

* Apply suggestions from code review

Co-authored-by: Lucain <[email protected]>
Co-authored-by: Omar Sanseviero <[email protected]>

* Delete Huggy + update load and push to hf

* Delete Huggy config file
* Update load_from_hf
* Update push_to_hf

* Apply suggestions from code review

Co-authored-by: Omar Sanseviero <[email protected]>
Co-authored-by: Lucain <[email protected]>

* Update push_to_hf.py

* Update with Omar Feedback

* Black style formatter

* Create __init__.py

* Updates based on flake8 review

* Change logging to MLAgents logger

* Update python version

* Update Python Version

* Update setup.py

* Update logger

* Update logger

* Pre-commit

* Update Tuple

* Update metadata generation

* Ignore mypy error

* Create Hugging-Face-Integration.md

* Update ML-Agents-Toolkit-Documentation.md

* Update style of Hugging-Face-Integration.md

* Typo

* Remove spaces

---------

Co-authored-by: Lucain <[email protected]>
Co-authored-by: Omar Sanseviero <[email protected]>

Author: 3 people

DevProject/Assets/csc.rsp

@@ -1,2 +1,2 @@
 -warnaserror+
--warnaserror-:618
+-warnaserror-:618

DevProject/DevProject.sln.DotSettings

@@ -1,2 +1,2 @@
 <wpf:ResourceDictionary xml:space="preserve" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:s="clr-namespace:System;assembly=mscorlib" xmlns:ss="urn:shemas-jetbrains-com:settings-storage-xaml" xmlns:wpf="http://schemas.microsoft.com/winfx/2006/xaml/presentation">
-	<s:String x:Key="/Default/CodeStyle/Naming/CSharpNaming/Abbreviations/=RL/@EntryIndexedValue">RL</s:String></wpf:ResourceDictionary>
+	<s:String x:Key="/Default/CodeStyle/Naming/CSharpNaming/Abbreviations/=RL/@EntryIndexedValue">RL</s:String></wpf:ResourceDictionary>

DevProject/ProjectSettings/Packages/com.unity.testtools.codecoverage/Settings.json

@@ -20,4 +20,4 @@
             }
         ]
     }
-}
+}

Project/Assets/ML-Agents/Examples/PushBlockWithInput/PushBlockActions.inputactions

@@ -176,4 +176,4 @@
             ]
         }
     ]
-}
+}

Project/Assets/ML-Agents/Examples/SharedAssets/Materials/ShaderOutline.shader

@@ -85,4 +85,4 @@ Shader "Custom/Outline and ScreenSpace texture"
 			ENDCG
 		}
 	}
-}
+}

Project/Assets/ML-Agents/Examples/SharedAssets/Materials/ShaderRadialGrid.shader

@@ -85,4 +85,4 @@
 // 			ENDCG
 // 		}
 // 	}
-// }
+// }

Project/Project.sln.DotSettings

@@ -1,3 +1,3 @@
 <wpf:ResourceDictionary xml:space="preserve" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:s="clr-namespace:System;assembly=mscorlib" xmlns:ss="urn:shemas-jetbrains-com:settings-storage-xaml" xmlns:wpf="http://schemas.microsoft.com/winfx/2006/xaml/presentation">
 	<s:String x:Key="/Default/CodeStyle/Naming/CSharpNaming/Abbreviations/=ML/@EntryIndexedValue">ML</s:String>
-	<s:Boolean x:Key="/Default/UserDictionary/Words/=Dont/@EntryIndexedValue">True</s:Boolean></wpf:ResourceDictionary>
+	<s:Boolean x:Key="/Default/UserDictionary/Words/=Dont/@EntryIndexedValue">True</s:Boolean></wpf:ResourceDictionary>

com.unity.ml-agents.extensions/.gitignore

@@ -13,4 +13,4 @@ build.bat.meta
 
 # API Scraper output
 *.api
-*.api.meta
+*.api.meta

com.unity.ml-agents.extensions/Runtime/Input/Unity.ML-Agents.Extensions.Input.asmdef

@@ -20,4 +20,4 @@
         }
     ],
     "noEngineReferences": false
-}
+}

com.unity.ml-agents.extensions/Tests/Runtime/Input/Unity.ML-Agents.Extensions.Input.Tests.Runtime.asmdef

@@ -28,4 +28,4 @@
         }
     ],
     "noEngineReferences": false
-}
+}

com.unity.ml-agents/CODEOWNERS

@@ -1,2 +1,2 @@
 # see https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/about-code-owners for more information
-* @unity/behavior-authoring
+* @unity/behavior-authoring

com.unity.ml-agents/Plugins/IL2CPP.DL.Stubs.c

@@ -7,4 +7,4 @@ void * dlopen(const char *filename, int flags) {
 
 void * dlsym(void *handle, const char *symbol) {
     return 0;
-}
+}

com.unity.ml-agents/README.md

@@ -12,4 +12,4 @@ Please refer to the [ML-Agents documentation] page for usage guides.
 
 
 [ML-Agents github repo]: https://github.com/Unity-Technologies/ml-agents
-[ML-Agents documentation]: https://unity-technologies.github.io/ml-agents/
+[ML-Agents documentation]: https://unity-technologies.github.io/ml-agents/

com.unity.ml-agents/Runtime/Grpc/Unity.ML-Agents.CommunicatorObjects.asmdef

@@ -15,4 +15,4 @@
     "defineConstraints": [],
     "versionDefines": [],
     "noEngineReferences": false
-}
+}

com.unity.ml-agents/Runtime/Unity.ML-Agents.asmdef

@@ -35,4 +35,4 @@
         }
     ],
     "noEngineReferences": false
-}
+}

com.unity.ml-agents/com.unity.ml-agents.sln.DotSettings

@@ -18,4 +18,4 @@
 	<s:Boolean x:Key="/Default/UserDictionary/Words/=Scriptable/@EntryIndexedValue">True</s:Boolean>
 	<s:Boolean x:Key="/Default/UserDictionary/Words/=statsd/@EntryIndexedValue">True</s:Boolean>
 	<s:Boolean x:Key="/Default/UserDictionary/Words/=stddev/@EntryIndexedValue">True</s:Boolean>
-	<s:Boolean x:Key="/Default/UserDictionary/Words/=vals/@EntryIndexedValue">True</s:Boolean></wpf:ResourceDictionary>
+	<s:Boolean x:Key="/Default/UserDictionary/Words/=vals/@EntryIndexedValue">True</s:Boolean></wpf:ResourceDictionary>

config/sac/FoodCollector.yaml

@@ -25,4 +25,4 @@ behaviors:
     max_steps: 2000000
     time_horizon: 64
     summary_freq: 60000
-    threaded: false
+    threaded: false

docs/Hugging-Face-Integration.md

@@ -0,0 +1,56 @@
+# The Hugging Face Integration
+
+The [Hugging Face Hub 🤗](https://huggingface.co/models?pipeline_tag=reinforcement-learning) is a central place **where anyone can share and download models**.
+
+It allows you to:
+- **Host** your trained models.
+- **Download** trained models from the community.
+- Visualize your agents **playing directly on your browser**.
+
+You can see the list of ml-agents models [here](https://huggingface.co/models?library=ml-agents).
+
+We wrote a **complete tutorial to learn to train your first agent using ML-Agents and publish it to the Hub**:
+
+- A short tutorial where you [teach **Huggy the Dog to fetch the stick** and then play with him directly in your browser](https://huggingface.co/learn/deep-rl-course/unitbonus1/introduction)
+- A [more in-depth tutorial](https://huggingface.co/learn/deep-rl-course/unit5/introduction)
+
+## Download a model from the Hub
+
+You can simply download a model from the Hub using `mlagents-load-from-hf`.
+
+You need to define two parameters:
+
+- `--repo-id`: the name of the Hugging Face repo you want to download.
+- `--local-dir`: the path to download the model.
+
+For instance, I want to load the model with model-id "ThomasSimonini/MLAgents-Pyramids" and put it in the downloads directory:
+
+```sh
+mlagents-load-from-hf --repo-id="ThomasSimonini/MLAgents-Pyramids" --local-dir="./downloads"
+```
+
+## Upload a model to the Hub
+
+You can simply upload a model to the Hub using `mlagents-push-to-hf`
+
+You need to define four parameters:
+
+- `--run-id`: the name of the training run id.
+- `--local-dir`: where the model was saved
+- `--repo-id`: the name of the Hugging Face repo you want to create or update. It’s always <your huggingface username>/<the repo name> If the repo does not exist it will be created automatically
+- `--commit-message`: since HF repos are git repositories you need to give a commit message.
+
+For instance, I want to upload my model trained with run-id "SnowballTarget1" to the repo-id: ThomasSimonini/ppo-SnowballTarget:
+
+```sh
+  mlagents-push-to-hf --run-id="SnowballTarget1" --local-dir="./results/SnowballTarget1" --repo-id="ThomasSimonini/ppo-SnowballTarget" --commit-message="First Push"
+```
+
+## Visualize an agent playing
+
+You can watch your agent playing directly in your browser (if the environment is from the [ML-Agents official environments](Learning-Environment-Examples.md))
+
+- Step 1: Go to https://huggingface.co/unity and select the environment demo.
+- Step 2: Find your model_id in the list.
+- Step 3: Select your .nn /.onnx file.
+- Step 4: Click on Watch the agent play

docs/ML-Agents-Toolkit-Documentation.md

@@ -35,6 +35,10 @@
 - [Creating Custom Side Channels](Custom-SideChannels.md)
 - [Creating Custom Samplers for Environment Parameter Randomization](Training-ML-Agents.md#defining-a-new-sampler-type)
 
+## Hugging Face Integration
+
+- [Using Hugging Face to download and upload trained models](Hugging-Face-Integration.md)
+
 ## Python Tutorial with Google Colab
 
 - [Using a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release_20_docs/colab/Colab_UnityEnvironment_1_Run.ipynb)

docs/Python-Custom-Trainer-Plugin.md

@@ -48,4 +48,4 @@ mlagents-learn ml-agents-trainer-plugin/mlagents_trainer_plugin/a2c/a2c_3DBall.y
 ## Tutorial
 Here’s a step-by-step [tutorial](Tutorial-Custom-Trainer-Plugin.md) on how to write a setup file and extend ml-agents trainers, optimizers, and
 hyperparameter settings.To extend ML-agents classes see references on
-[trainers](Python-On-Off-Policy-Trainer-Documentation.md) and [Optimizer](Python-Optimizer-Documentation.md).
+[trainers](Python-On-Off-Policy-Trainer-Documentation.md) and [Optimizer](Python-Optimizer-Documentation.md).

localized_docs/KR/docs/Training-Imitation-Learning.md

@@ -82,4 +82,4 @@ without pre-recording a demonstration file. The steps to do this are as follows:
 
 ## 한글 번역
 
-해당 문서의 한글 번역은 [민규식 (Kyushik Min)]([https://github.com/Kyushik](https://github.com/Kyushik))에 의해 진행되었습니다. 내용상 오류나 오탈자가 있는 경우 [email protected] 으로 연락주시면 감사드리겠습니다.
+해당 문서의 한글 번역은 [민규식 (Kyushik Min)]([https://github.com/Kyushik](https://github.com/Kyushik))에 의해 진행되었습니다. 내용상 오류나 오탈자가 있는 경우 [email protected] 으로 연락주시면 감사드리겠습니다.

localized_docs/KR/docs/Training-PPO.md

@@ -148,4 +148,4 @@ PPO는 에이전트의 관측 (Observation)을 통해 에이전트가 주어진
 
 ## 한글 번역
 
-해당 문서의 한글 번역은 [민규식 (Kyushik Min)]([https://github.com/Kyushik](https://github.com/Kyushik))에 의해 진행되었습니다. 내용상 오류나 오탈자가 있는 경우 [email protected] 으로 연락주시면 감사드리겠습니다.
+해당 문서의 한글 번역은 [민규식 (Kyushik Min)]([https://github.com/Kyushik](https://github.com/Kyushik))에 의해 진행되었습니다. 내용상 오류나 오탈자가 있는 경우 [email protected] 으로 연락주시면 감사드리겠습니다.

localized_docs/TR/docs/Getting-Started.md

@@ -175,4 +175,4 @@ Eğitimli modeliniz `results/<run-identifier>/<behavior_name>.nn` konumunda olac
   [Example Environments](Learning-Environment-Examples.md) page.
 - For more information on the various training options available, check out the
   [Training ML-Agents](Training-ML-Agents.md) page.
--->
+-->

ml-agents-envs/colabs/Colab_PettingZoo.ipynb

@@ -315,4 +315,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 4
-}
+}

ml-agents/mlagents/utils/__init__.py

@@ -0,0 +1,49 @@
+import os
+import argparse
+
+from huggingface_hub import snapshot_download
+
+from mlagents_envs import logging_util
+from mlagents_envs.logging_util import get_logger
+
+logger = get_logger(__name__)
+logging_util.set_log_level(logging_util.INFO)
+
+
+def load_from_hf(repo_id: str, local_dir: str) -> None:
+    """
+    Download a model from Hugging Face Hub.
+    :param repo_id: id of the model repository from the Hugging Face Hub
+    :param local_dir: local destination of the repository
+    """
+    _, repo_name = repo_id.split("/")
+
+    local_dir = os.path.join(local_dir, repo_name)
+
+    snapshot_download(repo_id=repo_id, local_dir=local_dir)
+
+    logger.info(f"The repository {repo_id} has been downloaded to {local_dir}")
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "--repo-id",
+        help="Repo id of the model repository from the Hugging Face Hub",
+        type=str,
+    )
+    parser.add_argument(
+        "--local-dir",
+        help="Local destination of the repository",
+        type=str,
+        default="./",
+    )
+    args = parser.parse_args()
+
+    # Load model from the Hub
+    load_from_hf(args.repo_id, args.local_dir)
+
+
+# For python debugger to directly run this script
+if __name__ == "__main__":
+    main()