[BugFix] Fix slow and flaky non-tensor parallel env test

[vmoens]

Stack from ghstack (oldest at bottom):

• -> [BugFix] Fix slow and flaky non-tensor parallel env test #2926
• [Refactor] Refactor the weight update logic #2914

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2926

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 Cancelled Job, 30 Pending, 1 Unrelated Failure

As of commit cb070c2 with merge base 21ef725 ():

CANCELLED JOB - The following job was cancelled. Please retry:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Unit-tests on Windows / unittests-cpu (3.10, windows.4xlarge, cpu) / windows-job (gh) (trunk failure)
  Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.