Support multiple remotes for projects to support forking workflow

[amisevsk]

Is your feature request related to a problem? Please describe.

It's not very common to work on git projects with a single remote; at the very least, a common flow is to have

origin: <your fork>
upstream: <upstream fork>

but it's further useful to add close collaborators' forks as remotes in your local copy as well.

It would be useful if the devfile supported assigning multiple remotes to a project.

Describe the solution you'd like

Option 1

Add remotes section

spec:
  projects:
    - name: project   
      git:
        location: "https://github.com/amisevsk/web-nodejs-sample.git"
      remotes:
        upstream: "https://github.com/che-samples/web-nodejs-sample.git"
        gh-collaborator: "https://github.com/gh-collaborator/web-nodejs-sample.git"        

Option 2

Support multiple locations instead of a plain string:

spec:
  projects:
    - name: project   
      git:
        location: # alt: 'remotes' ?
          origin: "https://github.com/amisevsk/web-nodejs-sample.git"
          upstream: "https://github.com/che-samples/web-nodejs-sample.git"
          gh-collaborator: "https://github.com/gh-collaborator/web-nodejs-sample.git"  
        primary: origin # optional; defines which fork should be checked out from      

Option 3

(added from #104 (comment) below)

spec:
  projects:
    - name: project   
      git:
        branch: foo
           remote: upstream # optional, defaults to origin
        remotes:
          origin: "https://github.com/amisevsk/web-nodejs-sample.git"
          upstream: "https://github.com/che-samples/web-nodejs-sample.git"
          gh-collaborator: "https://github.com/gh-collaborator/web-nodejs-sample.git"  

Additional context

Discussion in Eclipse Che issues: eclipse-che/che#17449

[metacosm]

Maybe something like:

spec:
  projects:
    - name: project   
      git:
        branch: foo
           remote: upstream # optional, defaults to origin
        remotes:
          origin: "https://github.com/amisevsk/web-nodejs-sample.git"
          upstream: "https://github.com/che-samples/web-nodejs-sample.git"
          gh-collaborator: "https://github.com/gh-collaborator/web-nodejs-sample.git"  

[l0rd]

I like @metacosm proposal. @amisevsk wdyt?

[amisevsk]

👍 LGTM

[benoitf]

what happens when we fork a repository with a devfile in it ? looks like paths/remotes won't follow.

could we have an option to automatically grab the remotes when it's github ? because we know with github if a repo is a fork from another one, etc.

so I just need to have a very simple devfile in the main repo and all forks will have this info automatically provided for github ?

[amisevsk]

One feature that we currently use in Che is that project can be left blank, which is taken to mean "import the repository this devfile belongs to". This allows us to have one devfile that will work for all forks automatically (which I think is partly what @benoitf is getting at). I don't know if this belongs in the devfile spec or should be left to implementation, as it has the downside of making devfiles technically not standalone. For full context, Che will accept either a) a devfile.yaml that is used to create a workspace or b) a link to a git(hub) repo from which it'll search for a devfile and fill the project with the git repo linked.

If we want to include this functionality in the spec, of the three proposals above, it's is easiest to do with Option 1: we could treat .spec.projects[].git as optional, and fill this with the repo that contains the devfile.

If we want to leave this up to implementation, option 3 works well, as we could potentially have the semantic

If origin is not specified in the remotes, it is assumed to be the repo containing the devfile.

or something similar.

Either way, I think having some sort of feature in devfiles where the same devfile can be used to work on a fork of a project naturally would be very useful. I'm currently maintaining devfiles separately from the main project's devfile to support working from my fork.

[sleshchenko]

@metacosm If it a valid yaml?

spec:
  projects:
    - name: project   
      git:
        init: # parameters that defined repository state after clonning
          branch: foo # optional, defaults to default branch
            remote: upstream # optional, defaults to origin

Some validators say no, some yes.
How it would look like in the JSON? Like the following?

{
   "spec": {
      "projects": [
         {
            "name": "project",
            "git": {
               "branch": "foo\nremote: upstream",
               "remotes": {
                  "origin": "https://github.com/amisevsk/web-nodejs-sample.git",
                  "upstream": "https://github.com/che-samples/web-nodejs-sample.git",
                  "gh-collaborator": "https://github.com/gh-collaborator/web-nodejs-sample.git"
               }
            }
         }
      ]
   }
}

[metacosm]

@sleshchenko indeed, it might not be…
Something like, then:

spec:
  projects:
    - name: project   
      git:
        init: # parameters that defined repository state after cloning
          branch: # optional, defaults to default branch
            name: foo # if a branch child is provided then a name for it must be provided as well
            remote: upstream # optional, defaults to origin

A little bit more verbose but more valid :)

[sleshchenko]

I would set

remote: upstream # optional, 
                 # defaults to origin if remotes are not specified or origin in the remotes list. 
                 # otherwise ??? (could be the first remote or required)

[sleshchenko]

Final format to discuss:
Simplest case

spec:
  projects:
    - name: project   
      git:
        remotes:
          origin: "https://github.com/amisevsk/web-nodejs-sample.git"

Multiple remotes

spec:
  projects:
    - name: project   
      git:
        remotes:
          origin: "https://github.com/amisevsk/web-nodejs-sample.git"
          upstream: "https://github.com/che-samples/web-nodejs-sample.git"
          gh-collaborator: "https://github.com/gh-collaborator/web-nodejs-sample.git"  
        cloneFrom: # mandatory if there is more than one remote
          branch: foo # mandatory
          remote: upstream # mandatory if there is more than one remote

[sleshchenko]

^ update:
cloneFrom: #optional -> cloneFrom: # mandatory if there is more than one remote

cc @metacosm @l0rd @amisevsk

[sleshchenko]

Git and github project source format is changed in the #120 to the following

projects:
  - name: project   
    git:
      remotes:
        origin: "https://github.com/amisevsk/web-nodejs-sample.git"
        upstream: "https://github.com/che-samples/web-nodejs-sample.git"
        gh-collaborator: "https://github.com/gh-collaborator/web-nodejs-sample.git"  
      checkoutFrom: # mandatory if there is more than one remote
        revision: foo # The revision that should be used checked out. May be branch name, tag or commit id.
                      # Default branch should be checked out if missing.
        remote: upstream # The remote name should be used as init. Required if there are more than one remote configured