Gopkg.toml in subdirectory of git project should not try to vendor other directories in same git project

[coriolinus]

Given a repository which contains a cmd folder which builds some number of executables, and an arbitrarily large number of other folders outside of cmd intended to be imported by other packages as libraries. In order to follow the principle that binaries handle vendoring, and libraries do not, it is desirable to always run dep ensure within the cmd folder. This ensures that Gopkg.* and vendor all appear there, so that external applications may depend on the libraries without encountering the constraints of the binaries.

dep handles this case poorly: the library portions of the repo, because they are not subfolders of the CWD in which dep is being run, are treated as external libraries and vendored. This is not desirable: a single repository should be considered to always be up-to-date with itself.

Example

$ tree -L 1
Mon Mar  4 12:09:11 CET 2019
.
├── cmd
├── crypto
├── libs
└── version

cmd contains a number of commands. crypto, libs, and version contain libraries we desire to export to other packages. When Gopkg.toml exists within cmd/, everything in those libraries is considered an external package, and must have a version specified. This is not desirable: by default, the most recent tagged version will be vendored. This does not allow for continued development on a dev branch.

Putting Gopkg.toml at the repo root fixes the problem for this package, but it makes things worse for packages downstream, using these libraries: whatever their constraints are must now also coexist with the constraints of this project. If this project has a command which has a constraint incompatible with the downstream project's constraint set, then they cannot solve their dependency graph, even if that constraint doesn't apply to the library portion of the code at all.

Desired fix

dep currently assumes that everything in the directory containing Gopkg.toml and its subfolders is a single project, always up-to-date with itself. These folders are therefore not vendored. This assumption should be broadened: everything within a single repository should be considered a single project, always up-to-date with itself, and excluded from vendoring. The repository root can be found very simply in the most common repositories:

• git rev-parse --show-toplevel
• hg root
• etc

What version of dep are you using (dep version)?

$ dep version
Mon Mar  4 11:57:30 CET 2019
dep:
 version     : v0.5.0
 build date  : 2018-08-16
 git hash    : 224a564
 go version  : go1.10.3
 go compiler : gc
 platform    : darwin/amd64
 features    : ImportDuringSolve=false

[kevinburke]

I believe this is the same issue as #1775 and #1768. Suggestion in #1775 is to use an ignored flag in Gopkg.toml to ignore the current library.

[coriolinus]

This may be the same issue, but that fix doesn't work: using an ignored flag, dep complains that there are no non-ignored go files in the repo, and refuses to continue.

[kevinburke]

Does the ignored flag cover the cmd subdirectory as well?

[coriolinus]

Yes--I'm not sure how to write an ignored flag which covers all subdirectories (many more than in the example) except the cmd directory without enumerating them all, which is a pain. Hence the feature request: at best, the ignored flag is a workaround, not desired behavior.

[kevinburke]

One common solution is to nest libraries under a src or lib subdirectory.

I'm open to making the change you suggest but not sure if/how soon you could expect to see it, or how many users that would break.

[coriolinus]

Agree: putting all the library code into a single subdirectory would be best. However, the code for which I really want this feature is an open source project with both binaries and libraries. I think the maintainers might be open to a PR moving dep's files into cmd, assuming that it just worked. I do not think that they would be willing to accept a PR which moved all their public libraries, breaking the code of everyone downstream. I'm aware that the only fast way to ensure progress is to open a PR here doing the work myself, and that absent that, I have to wait for a maintainer to find the time. That's fine. I'm glad that you agree that this is a reasonable feature to add. In the short term, that was my goal.

…

On Mon, Mar 4, 2019, 18:29 Kevin Burke ***@***.***> wrote: One common solution is to nest libraries under a src or lib subdirectory. I'm open to making the change you suggest but not sure if/how soon you could expect to see it, or how many users that would break. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2116 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHdeTij_JzfvtdaCOTQ-qx1EaTuWtP1Cks5vTVgCgaJpZM4bcIf_> .

[mvdan]

Dep was officially deprecated earlier this year, and the proposal to archive this repository was accepted. As such, I'm closing outstanding issues before archiving the repository. For any further comments, please use the proposal thread on the Go issue tracker. Thanks!