Integrating Hugging Face Hub 🤗 (updated) #5856

simoninithomas · 2023-01-03T12:19:28Z

Hey there 👋 ,

Updated version of #5732

Proposed change(s)

Integrating Hugging Face Hub 🤗 in ML-Agents to allow users to download and upload models to the Hub.
The difference with Integrating Hugging Face Hub 🤗 #5732 is that we updated the Hugging Face API process to push to the Hub + some cleanup.

An example of a model card and the tensorboard dashboard integrated in the Hub: https://huggingface.co/ThomasSimonini/testpyramidsrndNewintegration

A notebook to test the feature: https://colab.research.google.com/drive/1qcdo3LEA5H1f3Ds4MWFfM4mgCWUPWqzj?usp=sharing

Here's the process

You train your agent mlagents-learn --config="huggy.yaml" --run-id="Huggy_1"
After the training is done you push your model to the 🤗 Hub with mlagents-push-to-hf --run-id="Huggy_1" --local-dir="results/Huggy_1" --repo-id="ThomasSimonini/Huggy_test" --commit-message="Huggy upload"
Someone else wants to download it, it just need to type mlagents-load-from-hf --repo-id="ThomasSimonini/Huggy_test" --local-dir="./downloads"

What features do we want to add:

1️⃣ Add an evaluation step before pushing to the Hub the model

So I checked Statswriter but I have some thoughts and questions about the complete push to hub process design.
So people are going to train their model with mlagents-learn
After that to be able to evaluate the model they need to run mlagents-learn "folder containing the onnx" --resume --inference.
Because we created a new StatWriter we get the reward or elo and create a results.json put in the results folder and the result in the Hugging Face model card.
After that they will be able to push to hub using utils.push-to-hf
For the 2 instead of asking to write the command I suppose we can use subprocess (to run mlagents-learn "folder containing onnx" --resume --inference with specific plugin) in utils.push-to-hf?
The problem of the 2 comes from:

a. The fact that the number of testing timesteps == summary_freq => so it can be big. Do we have a solution for that or using episode instead?
b. Plus how to stop the inference? When it runs for one summary freq?

The very short-term solution could be instead, taking the final checkpoint in training_status.json directly on utils.push_to_hub.hf that contains either ELO or mean_reward (not std though).

2️⃣ Generate a video of the agent

For me, this is a good idea, but it might be a second PR.

Types of change(s)

Checklist

Added tests that prove my fix is effective or that my feature works
Updated the changelog (if applicable)
Updated the documentation (if applicable)
Updated the migration guide (if applicable)

Other comments

Remove use_auth_token

CLAassistant · 2023-01-03T12:19:38Z

All committers have signed the CLA.

*Change loading to snapshot_download (able to use cache)

* Use create_repo and upload_folder instead of git

simoninithomas · 2023-04-26T17:19:59Z

Hey there 👋 here is the integration update:

From outside nothing changed like the 3000 models already in the Hugging Face Hub, we run mlagents-push-to-hf --run-id --local-dir --repo-id --commit-message to upload a model and mlagents-load-from-hf --repo-id --local-dir to load a model.

The models: https://huggingface.co/models?library=ml-agents

We changed the upload system by using create_repo and upload_folder.
We changed the model card you can see an example of it here:
https://huggingface.co/ThomasSimonini/testpyramidsrndNewintegration

If you want to try the implementation I wrote a notebook (start at START HERE 😄 ): https://colab.research.google.com/drive/1omWRS5ArBC6kkyOV_-oG54Iz3BORvAsO?usp=sharing

Some questions:

Do you want to use a logger or print for the info given to the user during the downloading and the uploading?
Omar for the model card do you have something you want me to change?
Omar for setup.py I use "huggingface_hub~=0.8",.
I have a problem specific to Colab: ImportError: cannot import name 'notf' from 'tensorboard.compat' (/usr/local/lib/python3.9/dist-packages/tensorboard/compat/init.py)
What solutions do we have for that? I suppose it's due to Colab that now uses Python 3.9

🤗

Wauplin

Nice implementation @simoninithomas! I reviewed the logic and mostly made cosmetic comments. I haven't tested it myself as I was more focused on feedback about the huggingface_hub integration.

ml-agents/mlagents/utils/load_from_hf.py

ml-agents/mlagents/utils/push_to_hf.py

ml-agents/setup.py

ml-agents/mlagents/utils/push_to_hf.py

osanseviero

Very cool! I left some minor suggestions but this is looking great!

ml-agents/mlagents/utils/load_from_hf.py

ml-agents/mlagents/utils/push_to_hf.py

Co-authored-by: Lucain <[email protected]> Co-authored-by: Omar Sanseviero <[email protected]>

* Delete Huggy config file * Update load_from_hf * Update push_to_hf

Co-authored-by: Omar Sanseviero <[email protected]> Co-authored-by: Lucain <[email protected]>

osanseviero

Very cool! Thanks a lot!

ml-agents/mlagents/utils/push_to_hf.py

ml-agents/setup.py

simoninithomas · 2023-05-08T15:34:01Z

Updated ✅ what do you prefer for logging? Usually, I like to print messages so that people can see them on their console or notebook but logging is also a correct solution. 🤔

simoninithomas · 2023-05-15T14:19:37Z

~~Edit ✅ We changed the logger to MLAgents logger. However, the MLAgents logger does not display our info messages on collab. I'm wondering why 🤔 ?~~

Edit thanks to @osanseviero for the solution 🤗

The integration is ready feel free to tell me if you want us to change something 🤗

simoninithomas · 2023-05-30T12:13:39Z

All checks passed 🥳 are we ready to merge? Or do you need some other changes before?

miguelalonsojr

Seems like there's been many files touched without changes other than removing empty lines. Is there a reason for this? Did you rebase with develop on the main repo? Otherwise, the changes look good. The only request is to add a new documentation page that describes the new HF features and give some examples. You can add it to the docs folder and call it Hugging-Face-Integration.md. You'll also need to add a new section to the documentation overview https://github.com/Unity-Technologies/ml-agents/blob/develop/docs/ML-Agents-Toolkit-Documentation.md called Hugging Face Integration and then add a reference to the new integration md doc. You can follow the same format that's already in there. Once that's ready, I'll approve the PR and we can merge. Almost at the finish line!

simoninithomas · 2023-06-02T09:44:51Z

I've just added the documentation and run black let me know if you want me to change something else.

I don't think I rebase with develop on the main repo. HF-integration was created from develop because develop is the default branch in ML-Agents. Do I need to change something on this part?

miguelalonsojr · 2023-06-05T10:17:55Z

I don't think I rebase with develop on the main repo. HF-integration was created from develop because develop is the default branch in ML-Agents. Do I need to change something on this part?

You should rebase with the develop branch from the main repo. It may have diverged from the time that you created your fork/feature. I'll rerun the workflows and see if there are conflicts.

miguelalonsojr

LGTM.

simoninithomas added 7 commits January 3, 2023 11:24

Add Hugging Face Integration

ceea1d3

Update setup.py

e452d5f

Update push_to_hf.py

253c60c

Update push_to_hf.py

efa1c90

Update push_to_hf.py

eb7b671

Update push_to_hf.py

f99a9ec

Remove use_auth_token

Update push_to_hf.py

49aab55

simoninithomas changed the base branch from main to develop January 3, 2023 13:58

Update push_to_hf.py

9bfbdbf

osanseviero mentioned this pull request Feb 14, 2023

mlagents-push-to-hf doesn't exist huggingface/deep-rl-class#223

Closed

bterrific2008 mentioned this pull request Mar 13, 2023

Unit 5 issue with unity mlagents and python 3.9 compatibility in colab huggingface/deep-rl-class#236

Closed

Create Huggy

fa94f6d

osanseviero mentioned this pull request Apr 24, 2023

Integrating Hugging Face Hub 🤗 #5732

Closed

10 tasks

simoninithomas added 2 commits April 26, 2023 18:56

Update load_from_hf.py

f1c8b02

*Change loading to snapshot_download (able to use cache)

Update push_to_hf.py

d139693

* Use create_repo and upload_folder instead of git

Wauplin reviewed Apr 26, 2023

View reviewed changes

osanseviero reviewed Apr 26, 2023

View reviewed changes

simoninithomas and others added 4 commits April 27, 2023 10:41

Apply suggestions from code review

bd01072

Co-authored-by: Lucain <[email protected]> Co-authored-by: Omar Sanseviero <[email protected]>

Delete Huggy + update load and push to hf

6b3ad8d

* Delete Huggy config file * Update load_from_hf * Update push_to_hf

Apply suggestions from code review

8be1ae4

Co-authored-by: Omar Sanseviero <[email protected]> Co-authored-by: Lucain <[email protected]>

Update push_to_hf.py

c89cfae

osanseviero approved these changes Apr 27, 2023

View reviewed changes

ml-agents/mlagents/utils/push_to_hf.py Outdated Show resolved Hide resolved

ml-agents/mlagents/utils/push_to_hf.py Outdated Show resolved Hide resolved

ml-agents/mlagents/utils/push_to_hf.py Outdated Show resolved Hide resolved

ml-agents/setup.py Outdated Show resolved Hide resolved

simoninithomas added 3 commits April 28, 2023 10:59

Update with Omar Feedback

d0f1bd9

Black style formatter

c1ab298

Create __init__.py

e69b39c

simoninithomas added 2 commits May 8, 2023 19:08

Updates based on flake8 review

dbacbb6

Change logging to MLAgents logger

756f179

simoninithomas added 4 commits May 15, 2023 15:27

Update python version

ef0bf64

Update Python Version

719f714

Update setup.py

3ff2126

Update logger

b652fca

simoninithomas added 5 commits May 15, 2023 21:19

Update logger

b1c2008

Pre-commit

d7e41d8

Update Tuple

100dd3f

Update metadata generation

81a3c95

Ignore mypy error

67791c3

miguelalonsojr requested changes May 31, 2023

View reviewed changes

simoninithomas added 5 commits June 2, 2023 11:23

Create Hugging-Face-Integration.md

1dd2959

Update ML-Agents-Toolkit-Documentation.md

83d6bbe

Update style of Hugging-Face-Integration.md

ef7a887

Typo

b86eadc

Remove spaces

bfaba6c

miguelalonsojr approved these changes Jun 6, 2023

View reviewed changes

miguelalonsojr merged commit 0696876 into Unity-Technologies:develop Jun 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrating Hugging Face Hub 🤗 (updated) #5856

Integrating Hugging Face Hub 🤗 (updated) #5856

simoninithomas commented Jan 3, 2023

CLAassistant commented Jan 3, 2023 •

edited

Loading

simoninithomas commented Apr 26, 2023

Wauplin left a comment

osanseviero left a comment

osanseviero left a comment

simoninithomas commented May 8, 2023 •

edited

Loading

simoninithomas commented May 15, 2023 •

edited

Loading

simoninithomas commented May 30, 2023 •

edited

Loading

miguelalonsojr left a comment

simoninithomas commented Jun 2, 2023 •

edited

Loading

miguelalonsojr commented Jun 5, 2023

miguelalonsojr left a comment

Integrating Hugging Face Hub 🤗 (updated) #5856

Integrating Hugging Face Hub 🤗 (updated) #5856

Conversation

simoninithomas commented Jan 3, 2023

Proposed change(s)

Types of change(s)

Checklist

Other comments

CLAassistant commented Jan 3, 2023 • edited Loading

simoninithomas commented Apr 26, 2023

Wauplin left a comment

Choose a reason for hiding this comment

osanseviero left a comment

Choose a reason for hiding this comment

osanseviero left a comment

Choose a reason for hiding this comment

simoninithomas commented May 8, 2023 • edited Loading

simoninithomas commented May 15, 2023 • edited Loading

simoninithomas commented May 30, 2023 • edited Loading

miguelalonsojr left a comment

Choose a reason for hiding this comment

simoninithomas commented Jun 2, 2023 • edited Loading

miguelalonsojr commented Jun 5, 2023

miguelalonsojr left a comment

Choose a reason for hiding this comment

CLAassistant commented Jan 3, 2023 •

edited

Loading

simoninithomas commented May 8, 2023 •

edited

Loading

simoninithomas commented May 15, 2023 •

edited

Loading

simoninithomas commented May 30, 2023 •

edited

Loading

simoninithomas commented Jun 2, 2023 •

edited

Loading