Create a system for operating on local instances of repositories

[sdboyer]

We need to be able to operate on local instances of repositories, rather than relying on [the availability of] upstream hosting services for the running of our tests. There are three major reasons for this:

1. Test performance: the vast majority of time spent in our tests is exercising network behaviors. Having to actually go out over the network for each and every complete test run is an enormous waste of time. Reducing test running time would also incentivize adding more tests for more behaviors, which is certainly something we’d benefit from.
2. Test reliability: there’s the obvious possibility of erroneous test failures occurring as a result of an upstream hosting provider becoming temporarily unavailable. There’s also the less obvious possibility of failures arising from actual changes in the upstream source - e.g., if our test hardcodes the belief that a branch is at X commit, but it changes to Y, then we have a failure. We control for the latter by reusing a small set of repositories that I personally own and guarantee won’t change, but this limits our flexibility.
3. Test flexibility: some of the most important use cases and workflows for us to test involve how the tool behaves over the course of multiple actions when changes have occurred with upstream sources (new/[re]moved tags, new/[re]moved branches). While we can simulate this right now with the right lock file, it would be much, much better if we could model this directly, by actually making these upstream changes.

A bunch of work is required to make this a reality, including some work that’s likely to open up some interesting capabilities in other areas as we go. Here’s my current, rough sketch of what’s needed:

1. The SourceManager, and more specifically the pathDeducers, need to extend the set of schemes they allow to include file:///, or however that works for the relevant underlying vcs.
2. For testing purposes, we need to devise a mechanism for transparently rewriting inferred URLs over to some local scheme. This will allow real, valid import paths given in fixture .go files on disk to ultimately map to these file:///-local repositories. For this, we have some options:
   1. The absolute simplest implementation would be to create a SourceManager decorator that does on-the-fly rewrites to the ProjectIdentifier.Source property in all methods that take a ProjectIdentifier. (This approach would only really work for tests that don’t already use the ProjectIdentifier.Source property.)
   2. A more generally useful implementation would create a new subsystem within the sourceMgr for doing these on-the-fly rewrites. Such a system is actually crucial for some non-test use cases that we’ve known about for a while, but creates significant portability concerns (see Support for private repositories e.g. Github Enterprise #174).
3. We’d need to create a library of “upstream” sources for use in tests that we keep locally. I’m imagining a set of archives (uncompressed - git compresses for long-term storage and transport, no need to burn CPU on decompression during each test run), each of which contains e.g. an entire bare .git repository.
   1. To simulate upstream state changes but avoid non-determinism in test results, we’ll probably need to have distinct archives of the repositories at different states; let’s call these “source families.” (Other states in a family would have e.g. new or moved tags, new or moved branches, etc.)
   2. Each source family should have exactly one default state, which is
   3. Testing system abstractions should be created that allow switching between which of these states are targeted by calls in the test
   4. Tests that involve changing upstream states will need to be clearly delineated from tests that do not; the former can be performed using a shared SourceManager, but the latter will likely each have to have their own
   5. To assist in the creation and management of the source families, it’d be ideal to have a small helper tool that can be pointed at e.g. a git repository and archive/name/arrange/whatever it up appropriately

(This writeup is WIP, and will likely be updated/possibly moved into a spec doc)

[krisnova]

Do we need to create sub tasks for this larger epic? 🤔

[sdboyer]

@kris-nova yep, almost certainly. I really have no idea when I'd be able to find time/energy to do a more formal writeup of the requirements here, so it'd be ideal if folks could grab what seems to them like a chunk from the above, split it off into a separate issue, and then we can iteratively refine through dialogue.

[krisnova]

@sdboyer awesome! I will start cherry picking issues and opening up smaller GitHub from them.