Setup CI as running on TPU

[vfdev-5]

🚀 Feature

Ignite will support distributed training on TPU (e.g. #960). Currently, metric's computation is impacted in the same way as for DDP on GPUs. This should be addressed in a different Issue/PR.
Idea of this issue is to setup CI to emulate running on TPU as it is done in pytorch/xla.

• Setup another workflow on our CirlceCI as it is done for xla:

• https://github.com/pytorch/xla/blob/master/CONTRIBUTING.md#running-the-tests
• https://github.com/pytorch/xla/blob/master/.circleci/config.yml

• Add a simple test marked as TPU (@pytest.mark.tpu)

[erip]

I'd be interested in helping with this, but it seems like it'll require some administration on the CI side (setting env vars at least). Are TPU instances available freely for CI through CircleCI or is TPU virtualized through docker? Not sure how that works...

[vfdev-5]

@erip thanks ! I think it is CPU emulation what is done on xla CircleCI. If you could take a look how they propose contributors to work on xla dev and run tests, so, we can understand how to setup our tests. In our case, we wont need to rebuild xla etc, we can just use their docker and setup CPU emulation stuff.

but it seems like it'll require some administration on the CI side (setting env vars at least)

Let me check if I can activate CircleCI specific workflows on PR. But anyway, it is just about sending another .circleci/another_config.yml file, I think. Let me check. Otherwise, we can opt to Github acitons

EDIT: seems like we can have a single .circleci/config.yml. So, let's create this XLA CI workflow with Github Actions.

[erip]

It seems like if we want to use the XLA docker images that pytorch/pytorch and pytorch/xla use in GitHub Actions, we'll need to develop an action that wraps the container. What's not immediately clear is how the tests actually get run from there. 😄 I'll need to do some reading, but just commenting here to document for myself later.

[vfdev-5]

@erip I think it is more simple than that:

• run tests inside the docker like that : https://github.com/pytorch/ignite/blob/master/.circleci/config.yml#L83
• setup docker just by pulling their docker : docker pull gcr.io/tpu-pytorch/xla:r1.5