macos dep unable to find the import from vendor folder, but work well in window 10

[zigang93]

What version of dep are you using (dep version)?

dep:
version : devel
build date :
git hash :
go version : go1.9.1
go compiler : gc
platform : darwin/amd64

What dep command did you run?

dep ensure
dep ensure -update
dep status

This is my dep status

dep status
PROJECT CONSTRAINT VERSION REVISION LATEST PKGS USED
github.com/BurntSushi/toml ^0.3.0 v0.3.0 b26d9c3 b26d9c3 1
github.com/dmgk/faker branch master branch master 854b4c6 854b4c6 2
github.com/go-sql-driver/mysql ^1.3.0 v1.3 a0583e0 a0583e0 1
github.com/gorilla/context * v1.1 1ea2538 1ea2538 1
github.com/gorilla/mux ^1.6.1 v1.6.1 53c1911 53c1911 1
github.com/gorilla/schema branch master branch master afe7739 afe7739 1
github.com/gorilla/securecookie * v1.1 667fe4e 667fe4e 1
github.com/gorilla/sessions ^1.1.0 v1.1 ca9ada4 ca9ada4 1
github.com/jmoiron/sqlx branch master branch master de86474 de86474 2
github.com/josephspurrier/csrfbanana branch master branch master 2c49e35 2c49e35 1
github.com/justinas/alice branch master branch master 03f45bd 03f45bd 1
github.com/thedevsaddam/govalidator ^1.7.0 v1.7 ff1b1c7 ff1b1c7 1
golang.org/x/crypto branch master branch master a660000 a660000 2
gopkg.in/guregu/null.v3 ^3.3.0 v3.3.0 e81d6d8 e81d6d8 1

Explain

I try my best to explain what happen to my issue..
currently i am working on one project. sometime i switch between mac os and window pc.
when i run my go file, macos seem like cannot correct find the vendor folder.

example.go

import (
	"time"
	"strconv"
	"github.com/dmgk/faker"
	"gopkg.in/guregu/null.v3"
)

when i run go run example.go

terminal show this:

cmd/example.go:6:2: cannot find package "github.com/dmgk/faker" in any of:
        /usr/local/go/src/github.com/dmgk/faker (from $GOROOT)
        /Users/tanzigang/desktop/gocode/src/github.com/dmgk/faker (from $GOPATH)
cmd/example.go:7:2: cannot find package "gopkg.in/guregu/null.v3" in any of:
        /usr/local/go/src/gopkg.in/guregu/null.v3 (from $GOROOT)
        /Users/tanzigang/desktop/gocode/src/gopkg.in/guregu/null.v3 (from $GOPATH)

my vendor folder in my project:

It show already downloaded inside my vendor folder already.. you may also refer my dep status for double confirm.
but when i switch to window pc, all work well. that's weird..

[darkowlzz]

Hi, this looks more of related to go than dep. dep is doing the vendoring but go fails to find the package when you build it. Can you share the project or ways to reproduce this? I would be interested in knowing how are your project code organized. Code organization docs are here https://golang.org/doc/code.html#Organization .

[zigang93]

This is example how my project code organized.

bin/
    hello                          # command executable
    outyet                         # command executable
pkg/
    linux_amd64/
        github.com/golang/example/
            stringutil.a           # package object
src/
    github.com/golang/example/
        .git/                      # Git repository metadata
	hello/
	    hello.go               # command source
	outyet/
	    main.go                # command source
	    main_test.go           # test source
	stringutil/
	    reverse.go             # package source
	    reverse_test.go        # test source
    myproject        
         .git/                      # Git repository metadata

oh.. interesting ...
seem like go prefer github.com folder ?
can't just git clone the project folder inside the src folder?
so that make the macos problem? will update to u if this cause the problem.

[zigang93]

After i relocate my project in this structure.

bin/
    hello                          # command executable
    outyet                         # command executable
pkg/
    linux_amd64/
        github.com/golang/example/
            stringutil.a           # package object
src/
    github.com/golang/example/
        .git/                      # Git repository metadata
	hello/
	    hello.go               # command source
	outyet/
	    main.go                # command source
	    main_test.go           # test source
	stringutil/
	    reverse.go             # package source
	    reverse_test.go        # test source
    github.com/golang/myproject/        
         .git/

when i dep ensure
it show error below:
unable to deduce repository and source type
and unable to read metadata: go-import metadata not found

same happen as below issue:
#860

I think still need to wait PR to fix this.

[darkowlzz]

when i dep ensure
it show error below:
unable to deduce repository and source type
and unable to read metadata: go-import metadata not found

This error depends on the repository you're trying to vendor. If the repository no longer exists in the original location, or some firewall is blocking access to that repository, this happens. Or, if you don't have a go-import metadata server to serve the metadata of the repository, this happens.

Sharing your Gopkg.toml and the exact error would help.

[zigang93]

since I relocate the the working folder.
I just make a clean install.. vendor folder has been deleted.

my Gopkg.toml

[[constraint]]
  name = "github.com/BurntSushi/toml"
  version = "0.3.0"

[[constraint]]
  branch = "master"
  name = "github.com/dmgk/faker"

[[constraint]]
  name = "github.com/go-sql-driver/mysql"
  version = "1.3.0"

[[constraint]]
  name = "github.com/gorilla/mux"
  version = "1.6.1"

[[constraint]]
  branch = "master"
  name = "github.com/gorilla/schema"

[[constraint]]
  name = "github.com/gorilla/sessions"
  version = "1.1.0"

[[constraint]]
  branch = "master"
  name = "github.com/jmoiron/sqlx"

[[constraint]]
  branch = "master"
  name = "github.com/josephspurrier/csrfbanana"

[[constraint]]
  branch = "master"
  name = "github.com/justinas/alice"

[[constraint]]
  name = "github.com/thedevsaddam/govalidator"
  version = "1.7.0"

[[constraint]]
  branch = "master"
  name = "golang.org/x/crypto"

[[constraint]]
  name = "gopkg.in/guregu/null.v3"
  version = "3.3.0"

run dep status and dep ensure have same error:

The following errors occurred while deducing packages:
  * "pastryworld.co/app/model/service": unable to deduce repository and source type for "pastryworld.co/app/model/service": unable to read metadata: go-import metadata not found
  * "pastryworld.co/lib/route/middleware/login": unable to deduce repository and source type for "pastryworld.co/lib/route/middleware/login": unable to read metadata: go-import metadata not found
  * "pastryworld.co/lib/validator": unable to deduce repository and source type for "pastryworld.co/lib/validator": unable to read metadata: go-import metadata not found
  * "pastryworld.co/lib/view/plugin": unable to deduce repository and source type for "pastryworld.co/lib/view/plugin": unable to read metadata: go-import metadata not found
  * "pastryworld.co/lib/config": unable to deduce repository and source type for "pastryworld.co/lib/config": unable to read metadata: go-import metadata not found
  * "pastryworld.co/lib/view": unable to deduce repository and source type for "pastryworld.co/lib/view": unable to read metadata: go-import metadata not found
  * "pastryworld.co/app/model/entity": unable to deduce repository and source type for "pastryworld.co/app/model/entity": unable toread metadata: go-import metadata not found
  * "pastryworld.co/lib/session": unable to deduce repository and source type for "pastryworld.co/lib/session": unable to read metadata: go-import metadata not found
  * "pastryworld.co/app/controller": unable to deduce repository and source type for "pastryworld.co/app/controller": unable to read metadata: go-import metadata not found
  * "pastryworld.co/lib/util": unable to deduce repository and source type for "pastryworld.co/lib/util": unable to read metadata: go-import metadata not found
  * "pastryworld.co/lib/server": unable to deduce repository and source type for "pastryworld.co/lib/server": unable to read metadata: go-import metadata not found
  * "pastryworld.co/lib/database": unable to deduce repository and source type for "pastryworld.co/lib/database": unable to read metadata: go-import metadata not found
  * "pastryworld.co/lib/flash": unable to deduce repository and source type for "pastryworld.co/lib/flash": unable to read metadata: go-import metadata not found
  * "pastryworld.co/lib/route/middleware/super": unable to deduce repository and source type for "pastryworld.co/lib/route/middleware/super": unable to read metadata: go-import metadata not found
  * "pastryworld.co/lib/route/middleware/install": unable to deduce repository and source type for "pastryworld.co/lib/route/middleware/install": unable to read metadata: go-import metadata not found
  * "pastryworld.co/lib/route/middleware/logrequest": unable to deduce repository and source type for "pastryworld.co/lib/route/middleware/logrequest": unable to read metadata: go-import metadata not found
  * "pastryworld.co/lib/route": unable to deduce repository and source type for "pastryworld.co/lib/route": unable to read metadata: go-import metadata not found

validateParams: could not deduce external imports' project roots

I just realize the error is from my package import.. none of them come from Gopkg.toml, no vendor folder also.

What am i doing wrong??
extra info, my project is private repo.

[zigang93]

Update

Finally dep ensure is working now.

I just use ignored in Gopkg.toml

ignored = [
  "pastryworld.co/lib/validator", "pastryworld.co/lib/util" , "pastryworld.co/app/model/service", "pastryworld.co/lib/view" , 
  "pastryworld.co/app/controller" , "pastryworld.co/lib/route",
  "pastryworld.co/lib/route/middleware/install" , "pastryworld.co/lib/session" ,
  "pastryworld.co/app/model/entity" , "pastryworld.co/lib/config" , "pastryworld.co/lib/database" , "pastryworld.co/lib/view/plugin" , "pastryworld.co/lib/flash" , "pastryworld.co/lib/route/middleware/logrequest" ,
  "pastryworld.co/lib/server" , "pastryworld.co/lib/route/middleware/super" ,
  "pastryworld.co/lib/route/middleware/login"
]

extra info:
pastryworld.co is my project folder name.

What I learned:
Well, I just cannot understand that why macos need extra step/code to make the dep working?
in window 10, I can just git clone the project folder in go src folder and just run dep ensure , everything work fine and perfect. We also tried in linux. it also work well like window 10.

just the macos, the project folder need to be locate below this structure ,

bin/
pkg/
src/
    github.com/user/pastryworld.co/        
         .git/

if not follow the structure, will cause the cannot find package (from $GOROOT) problem.
Even it already installed in vendor folder after run dep ensure.

rather than this: (window and linux)

bin/
pkg/
src/
  pastryworld.co/        
         .git/

and also need to add ignored in Gopkg.toml. since in my .go file write import with pastryworld.co/xx rather than github/user/pastryworld.co/xx

or i should just write the github/user/pastryworld.co/xx in import
than just delete all the ignored in Gopkg.toml`.

[sdboyer]

It seems like there's some model mismatch happening with how you're setting up your project on disk, and naming your import paths. It's hard to say exactly where, but going through the intro guides may help.

One problem here appears to be that your import statements point to, for example, pastryworld.co/lib/validator, rather than github.com/user/pastryworld.co/lib/validator. dep has to figure out which import paths are internal - that is, correspond to packages within your project - vs. which are external. It does this by inferring your project's root import path by looking at the position of Gopkg.toml relative to GOPATH, then doing prefix checking of that string against the import paths it sees in your project.

To be clear, dep currently treats vendor/ as dead code. Its existence, or non-existence, doesn't affect the part of dep's behavior where it selects dependencies. As @darkowlzz indicated, dep needs to be able to figure out the network location for each project it interacts with in order to function.