Add an --exec option

[johnkeeping]

While git-imerge is great for identifying where textual conflicts are introduced, I would also like to be able to detect where logical conflicts are introduced.

At the moment this can be accomplished using the --manual mode and wrapping a script around make && git imerge continue, but it would be nice if git-imerge could be automated in the same way as git-bisect in order to check the consistency of the result even when no textual conflict has been found.

[mhagger]

Yes, that would be very nice. Do you already have some code that you are willing to share with us?

The obvious place to wire in a test would be in automerge(), though I haven't thought through whether it is as simple as that.

But I think that building and/or running tests will make the test merges so expensive that other inefficiencies will become very annoying (as for example you observed in issue #59). For example, presumably the build/test would be run during the test merges (e.g., the ones made during bisection), and then when autofilling the same merges are done (albeit from different parents). But, as long as the tree SHA-1s match, there is no reason to run the same build/tests again. For example, one could keep a record of trees that have tested OK, and if the same tree reappears, just assume that it is still OK.

[johnkeeping]

The patch I'm currently using is here. I opted to only run make when performing test merges and not for autofilling; so far that has always pointed me at the exact upstream commit (more often it's actually merge commit) that introduced the change that breaks my build.

I don't think I understand enough about autofilling to understand your comment on trees; I agree that if the same tree appears then there should not be a need to run the script again.

[mhagger]

Thanks for sharing the patch.

I opted to only run make when performing test merges and not for autofilling

Interesting; that was one of the things that I was wondering about.

And if you are only running the build when performing test merges, then probably the same tree is not being tested multiple times (simply because trees shouldn't be seen more than once when testing).

I don't expect to have time to work on this soon, so if somebody else is interested, by all means step up. Aside from @johnkeeping's patch (or something like it), I guess we would need a new option for the user to use to specify to the test command. The test command should probably be stored as a string to the git config and executed using the shell. And we might want to handle test command failures different than merge conflicts; perhaps there should be two separate exceptions, both derived from AutomaticMergeFailed but that emit different error messages.

[johnkeeping]

Are you suggesting the command is configured with git config? I had assumed that the command would need to be stored in the imerge state so that it can be set differently for different merges, but I can see how putting it in the config is nice and simple.

[mhagger]

Yes, I think the command should be stored using git config. (Of course by that I don't mean the user should have to type git config herself/himself!)

It would be wrong to store the command choice in the git repository in the imerge state blob, because (especially when we better support collaborations on imerges) it should not be possible for one user to cause another user to execute an arbitrary command (security risk). And it could very well be that different users want to run different commands for testing, or that their differing local environments require the commands to be run differently.

But we can nevertheless store it in a part of the configuration specific to a particular imerge:

[imerge "myname"]
        testCommand = "make -j16 test"

We might also offer a default setting that is used if there is no setting for a particular imerge (because, realistically, the command will probably be the same for different imerge runs within the same repository):

[imerge]
        testCommand = "make -j16 test"

If imerge were the only program involved, I'd stop there. But I've had a few ideas about how various commands could use tests. For example, git bisect can use a test, pre-commit hooks might want to run tests, one or two scripts of my own sometimes want to run tests. So I've been thinking that it would be nice to have a way to specify tests more abstractly within a repository and refer to them from different commands. Think of something like

[test "build"]
        command = make -j8
[test "fresh"]
        # These commands would be run in sequence; all must succeed:
        command = git clean -dfx
        command = make configure
        command = ./configure --prefix=$HOME
        command = make -j8
[test "full"]
        command = ionice -c 3 make -j8 test
[test]
        default = build
[imerge "myname"]
        test = full

Maybe there should even be a new command, git test, that runs tests that are configured in this way and memoizes the results (e.g., in git notes) for trees that have been tested.