proposal: Go 2: cmd/go: allow relative imports

[cch123]

I think this is a useful feature for library writers, sometimes dependency can not be avoided for a lib.
eg. a proxy lib may use a sql parser written by third party.

The library package layout may look like follows:

myproxy: host on github or sth
  main.go
  └── import "github.com/xxx/sqlparser"
  └── import "github.com/yyy/sometcppoolpackage"

If I want to make the dependency invisible to lib endusers, currently I have to move the dependencies to internal package, and manually change the import path to, maybe:

import "github.com/myname/dbproxy/internal/github.com/xxx/sqlparser"

It's quite ugly and inconvenient.

If internal can be imported easily, then import path will be,

import "internal/github.com/xxx/sqlparser"

quite clear

I noticed that some libs also have inner package, and they currently layout their project like the following:

mqtool is hosted on github.com/xxx/mqtool

mqtool/
  main.go
   └─ import "github.com/xxx/mqtool/cli"
  otherfile.go
  └─ import "github.com/xxx/mqtool/cli"
  └─ import "github.com/xxx/mqtool/logutil"
  ... many many file
  otherfileend.go
  └─ import "github.com/xxx/mqtool/bytes2"

If someone wants to fork this mqtool lib to fix bugs or maintain it as his own, then he will meet a lot problems (to modify endless import paths).

Someone may say that he can git clone this project to the correct path, but go get can not support such case, which is really weird (because go get will put this project to the forked path, not the original path).

If internal package can be found in relative path, I think things can go much better :-)

Then the above project may become like this:

mqtool is hosted on github.com/xxx/mqtool

mqtool/
  main.go
   └─ import "internal/cli"
  otherfile.go
  └─ import "internal/cli"
  └─ import "internal/logutil"
  ... many many file
  otherfileend.go
  └─ import "internal/bytes2"

[rsc]

This is not really about internal at all, since you could have the same situation with non-internal subdirectories you want to import. For better or worse we've decided that having fully-qualified paths is useful for being able to understand imports when you see them. It's not clear we want to break that now.

[leonklingele]

Can this be reopened as a proposal for Go 2 or should I create another proposal?

I'm especially asking for something similar to this:

mqtool is hosted on github.com/xxx/mqtool

mqtool/
  main.go
   └─ import "internal/cli"
  otherfile.go
  └─ import "internal/cli"
  └─ import "internal/logutil"
  ... many many file
  otherfileend.go
  └─ import "internal/bytes2"

It doesn't make sense to hard-code the platform / domain I'm going to host my code on inside the code itself.

[ianlancetaylor]

I'll reopen for Go 2.

[leonklingele]

While I'm not a fan of internal/.., something like this ("project-relative imports") should be considered for Go 2. internal could be a package name, can it? So this could potentially break existing code.
Instead, I'd prefer import "/x/y" (leading slash indicates project-relative import, / is the root of the project)

Hard-coding the platform / domain one intends to host code under should not be part of the code itself, moving code between different hosting providers (or even renaming the project) should be easier and should not require a.) changes to the source code itself (s/old-domain/new-domain/) b.) the use of a self-hosted go-import-providing server: example which gets the source from diverse hosting platforms but uses a single, static custom domain name.

This proposal basically is a requirement for proposal: incrementally modify the Go toolchain to work without GOPATH

[dlsniper]

I would discourage relative imports to begin with. They can lead to unexpected results. The current model of paths being GOPATH dependent works really well and can be adapted towards a package oriented workflow rather easily.

This proposal basically is a requirement for proposal: incrementally modify the Go toolchain to work without GOPATH

I would disagree about that.

[leonklingele]

I would discourage relative imports to begin with. They can lead to unexpected results

For example?

I would disagree about that.

Why? How else would the Go compiler toolchain know if an import should be retrieved from $GOPATH / vendor/ (i.e. external packages) or from the current project?

[johnDoe2018]

I'd share another example where this could be useful. Let's assume there is a project with the following structure:

---

github.com/janeRoe/myapp

• • main.go - imports models, handlers, and routes:

import (
	"github.com/janeRoe/myapp/handlers"
	"github.com/janeRoe/myapp/models"
	"github.com/janeRoe/myapp/routes"
)

• • handlers/
• • models/
• • routes/

---

I want to fork the project and make a few changes to, say, routes. I click GH "Fork" button and get a copy of the project that now lives under github.com/johnAnonDoe/myapp. But the main.go's imports still point to the old "github.com/janeRoe/myapp/...". That means the following requirements cannot be satisfied at the same time:

1. Changes to the routes/ subpackage must affect my fork (which, in case of absolute import paths, requires update of the main.go).
2. My fork should be go getable.
3. It should be easy to merge the changes back to the initial repo.

[ianlancetaylor]

Because currently Go packages are distinguished by import paths, support for relative import paths would mean that very early on in the build process the relative imports would have to be converted to full paths.

As far as I can see the only time it is useful to have a relative import path is when you are going to move a set of packages around. That seems to happen relatively rarely, and it would be straightforward to write a tool to move a set of packages while rewriting the import paths. I'm not aware of any such tool today, but if we had one would there still be a need for relative import paths?

[cch123]

@ianlancetaylor , if I fork other people's project on github, and want to contribute back to this project.It seems there is still no way to keep the forked project go gettable and the import paths inside this forked project to be same with the original's.

If I want to use it as my own, I have to rewrite the import path, and if I want to contribute, I have to rewrite the import path again. So I think it's not quite friendly to the current github workflow.(but if you can provide a rewriting tool, this annoying procedure can be more convenient than now.

[ianlancetaylor]

@cch123 Seems like you may be assuming that the other person's project is already using relative imports, which it most likely is not.

[cch123]

@ianlancetaylor ，for example，I forked etcd project，this file https://github.com/coreos/etcd/blob/master/discovery/discovery.go contains imports of the same project's paths:

	"github.com/coreos/etcd/client"
	"github.com/coreos/etcd/pkg/transport"
	"github.com/coreos/etcd/pkg/types"

After forked to my own repo, the import path should be:

	"github.com/cch123/etcd/client"
	"github.com/cch123/etcd/pkg/transport"
	"github.com/cch123/etcd/pkg/types"

If I don't change the import paths and then execute

go get github.com/cch123/etcd

Then the project will appear at $GOPATH/src/github.com/cch123/etcd

But the import paths inside this project are not correct. I'm not assuming other people is using relative imports, but I mean if we could support some way(eg.relative import? or some way of internal usage) to unbind the import path with the host address, then I will not be forced to change the import paths back and forth.

The original issue is that I want to make use of internal feature to bind dependencies together with my library. Because dependencies on site like github cannot avoid risks to be deleted by original author(like the sad story of leftpad in npm community). So I think feature of this could make my library more safe in such situation. But if I make these dependencies as my library internals , it will also meet the import path problem. I have to change all the import paths of these dependencies if they use the import path like the above case.

[dlsniper]

@cch123 I recommend you read http://blog.sgmansfield.com/2016/06/working-with-forks-in-go/ on how to work with forks if you want to contribute changes to a project. If you want to use your fork instead of the original place, dep supports this (and I think vgo may support this at some point, if not already).

Relative import paths is something I would love to see gone in Go2 as they are a constant source of pain.

[nkovacs]

I agree with @cch123 forking packages with subpackages in them is a huge pain.
Changing origins is not a solution (you can't go get them). Dep's support for forks is a band-aid.

You can actually abuse the vendor system to solve this problem. Just put your subpackages into vendor, and you've got project-relative imports, you can fork the package easily without having to rewrite everything. Obviously that's an ugly hack, but I'd like to see something like that as a solution to this problem, maybe not full relative imports.

[dlsniper]

I agree with @cch123 forking packages with subpackages in them is a huge pain.
Changing origins is not a solution (you can't go get them). Dep's support for forks is a band-aid.

@nkovacs this is not a band-aid, nor would relative import paths solve anything in this case. dep allows something that's considered basic functionality for package managers. There is nothing new or different for Go compared to other languages.

Just put your subpackages into vendor, and you've got project-relative imports, [...] Obviously that's an ugly hack

This is not why we have the vendor directory. We have it for reproducible builds. It allows guaranteed that, if committed, everyone gets the same identical source code regardless of environment.

but I'd like to see something like that as a solution to this problem, maybe not full relative imports.

I don't understand how relative imports solve any problem that you have.

[nkovacs]

If I fork a php package, I can publish it on packagist without first having to change a bunch of imports inside it.
If I fork a go package with subpackages, I have to change all the imports to its subpackages in it if I want people to go get it, or govendor fetch it, or dep ensure it, or whatever.

Both composer and npm allow you to specify a temporary fork instead of the canonical package. But in both ecosystems you can just publish your forked package without having to rewrite a bunch of imports inside it, because they're all relative. You can't do that in go.

This is not why we have the vendor directory. We have it for reproducible builds. It allows guaranteed that, if committed, everyone gets the same identical source code regardless of environment.

Of course not. I said you can abuse it to fix the problem.

I don't understand how relative imports solve any problem that you have.

I wouldn't have to do this: nkovacs/ifacemaker@c327e8d
Or this: nkovacs/counterfeiter@d0d3dfc

[dominikh]

If I fork a go package with subpackages, I have to change all the imports to its subpackages in it if I want people to go get it, or govendor fetch it, or dep ensure it, or whatever.

Which, compared to the effort of forking, modifying and maintaining a project, takes absolutely zero effort.

[dlsniper]

But in both ecosystems you can just publish your forked package without having to rewrite a bunch of imports inside it, because they're all relative. You can't do that in go.

To give you the counter-point for the PHP example. You would have to override the autoload process to tell it that your project provides now the reference for the namespace you want to use, rather than the original package, see https://github.com/Seldaek/monolog/blob/master/composer.json#L46-L48

Now Go being different than PHP is is allowed to behave in a different way. If you ask me, instructing dep to fetch the source from a different place is a lot more cleaner and obvious than digging in the package.json and figuring out where the autoloader will perform the operation for loading certain package.

And even in PHP or in Go world, allowing relative import paths suffer greatly when introducing symlinks into play (which at least for me in the PHP world have been a source of needlessly spent hours of debugging why something doesn't work in a particular setup of a developer / environment or in bug reports that I had to handle).

As far as I can tell, the whole argument is: please allow relative import paths so we don't have to run a search & replace operation for forks we want to use as main source but we don't want to use dep and we don't want to have a commit for that operation and we'll never backport changes to upstream (due to the import path change). Would this be an accurate depiction of the case?

[sebastianhaberey]

I, too, ran into the issues pointed out by @johnDoe2018 and @cch123 recently. I tried the proposed solution involving dep ensure, but I could not get it to work for internal packages. I tend to agree with @cch123 that this dep functionality is a bit of a band-aid. From the documentation:

source rules are generally brittle and should only be used when there is no other recourse.

I am convinced that there are good reasons to completely remove relative paths from Go and so I'll leave that discussion to people more familiar with Go. On the other hand I think that forking repositories, creating pull requests and validating them on CI systems are part of daily developer life. The current need to adress packages inside my own project by their full repository path seems to conflict with that workflow.

[tv42]

The underlying problem seems to be solved by go modules and their support for "github-forking etcd", with a git clone fetching whatever clone you want, while avoiding all the downsides of relative imports. And writing code using forked libraries is supported with replace directive in go.mod.

[surajbarkale]

How about treating import paths that start with / to be relative to the root of module? So the following code blocks are equivallant?

	"github.com/coreos/etcd/client"
	"github.com/coreos/etcd/pkg/transport"
	"github.com/coreos/etcd/pkg/types"

With module relative import

	"/client"
	"/pkg/transport"
	"/pkg/types"

Since the module name is present from go.mod, this will not require any significant changes to the import machinery.

[wadeperrigo]

I would say relative paths are extremely important whenever developing an application. If my understanding is correct from this conversation any "module" I create needs to be moved to the go PATH rather than simply referencing it from "./myModule" or "./myModule.go". This seems extremely inconvenient and while not being able to auto-fork or pull down from a github repository as my goal, but rather just use modules I make locally - it seems relative pathing would make life so much easier. (IE: Look at JS)

[ianlancetaylor]

With our current module ecosystem, our use of replace directives in go.mod files, the introduction of go.work files, the existence of the gomvpkg command, we're at a point where it's clear that are not going to start supporting relative imports. Closing this issue.

[awy]

It feels to me as though the proposal to use import paths starting with slash as same-module imports would be amenable to Go 1, at the very least by providing a suitable directive in go.mod if necessary. I don't see why this needs to be deferred to Go 2.

One would then have 3 styles of import:

• import pkg // standard Go library
• import domain/path/module/pkg // external module
• import /pkg // relative to root of current module

The module fork-then-PR use-case (or even fork without PRs) is the most persuading of the need for such functionality.

Renaming or relocating a module may be unusual in some patterns of use but overall, not sufficiently unusual that the pain need not be alleviated.