-
Notifications
You must be signed in to change notification settings - Fork 0
Home
根据提议,Go 1.11包括了对版本化模块的初步支持。模块是Go 1.11中的一个实验性选择加入的功能,并计划在Go 1.13整合反馈并最终确定功能。尽管某些细节可能会改变,但后续版本将支持使用Go 1.11或1.12定义的模块。
最初的原型(vgo
)于2018年2月发布。2018年7月,对版本化模块的支持合入主库。Go 1.11于2018年8月发布。
请通过现有提issues或 experience reports提供对于modules的反馈。
The "Quick Start" and "New Concepts" sections are particularly important for someone who is starting to work with modules. The "How to..." sections cover more details on mechanics. The largest quantity of content on this page is in the FAQs answering more specific questions; it can be worthwhile to at least skim the FAQ one-liners listed here.
- Quick Start
- New Concepts
- How to Use Modules
- Migrating to Modules
- Additional Resources
- Changes Since the Initial Vgo Proposal
- GitHub Issues
-
FAQs
- How are versions marked as incompatible?
- When do I get old behavior vs. new module-based behavior?
- Why does installing a tool via 'go get' fail with error 'cannot find main module'?
- How can I track tool dependencies for a module?
- What is the status of module support in IDEs, editors and standard tools like goimports, gorename, etc.?
-
FAQs — Additional Control
- What community tooling exists for working with modules?
- When should I use the 'replace' directive?
- Can I work entirely outside of VCS on my local filesystem?
- How do I use vendoring with modules? Is vendoring going away?
- Are there "always on" module repositories and enterprise proxies?
- Can I control when go.mod gets updated and when the go tools use the network to satisfy dependencies?
- How do I use modules with CI systems such as Travis or CircleCI?
-
FAQs — go.mod and go.sum
- Why does 'go mod tidy' record indirect and test dependencies in my 'go.mod'?
- Is 'go.sum' a lock file? Why does 'go.sum' include information for module versions I am no longer using?
- Should I still add a 'go.mod' file if I do not have any dependencies?
- Should I commit my 'go.sum' file as well as my 'go.mod' file?
-
FAQs — Semantic Import Versioning
- Why must major version numbers appear in import paths?
- Why are major versions v0, v1 omitted from import paths?
- What are some implications of tagging my project with major version v0, v1, or making breaking changes with v2+?
- Can a module consume a package that has not opted in to modules?
- Can a module consume a v2+ package that has not opted into modules? What does '+incompatible' mean?
- How are v2+ modules treated in a build if modules support is not enabled? How does "minimal module compatibility" work in 1.9.7+, 1.10.3+, and 1.11?
- What happens if I create a go.mod but do not apply semver tags to my repository?
- Can a module depend on a different version of itself?
-
FAQs — Multi-Module Repositories
- What are multi-module repositories?
- Should I have multiple modules in a single repository?
- Is it possible to add a module to a multi-module repository?
- Is it possible to remove a module from a multi-module repository?
- Can a module depend on an internal/ in another?
- Can an additional go.mod exclude unnecessary content? Do modules have the equivalent of a .gitignore file?
- FAQs — Minimal Version Selection
-
FAQs — Possible Problems
- What are some general things I can spot check if I am seeing a problem?
- What can I check if I am not seeing the expected version of a dependency?
- Why am I getting an error 'cannot find module providing package foo'?
- Why does 'go mod init' give the error 'cannot determine module path for source directory'?
- I have a problem with a complex dependency that has not opted in to modules. Can I use information from its current dependency manager?
- How can I resolve "parsing go.mod: unexpected module path" and "error loading module requirements" errors caused by a mismatch between import paths vs. declared module identity?
- Why does 'go build' require gcc, and why are prebuilt packages such as net/http not used?
- Do modules work with relative imports like
import "./subdir"
? - Some needed files may not be present in populated vendor directory
这里是一个从零开始创建模块的简单示例,详细信息将在本页的其余部分中介绍。
GOPATH目录外创建一个目录:
$ mkdir -p /tmp/scratchpad/hello
$ cd /tmp/scratchpad/hello
初始化一个新的module:
$ go mod init www.greatytc.com/you/hello
go: creating new go.mod: module www.greatytc.com/you/hello
编写代码:
$ cat <<EOF > hello.go
package main
import (
"fmt"
"rsc.io/quote"
)
func main() {
fmt.Println(quote.Hello())
}
EOF
构建并运行:
$ go build
$ ./hello
Hello, world.
go.mod
文件会更新,其中包含了依赖的显式版本,这里的v1.5.2
是一个语义化版本 tag:
$ cat go.mod
module www.greatytc.com/you/hello
require rsc.io/quote v1.5.2
注意,上面的示例中不需要go get
。
您的典型日常工作流如下:
- 根据需要将import语句添加到
.go
代码中。 - 标准命令如
go build
或go test
将根据需要自动添加新的依赖项以满足imports(更新go.mod
并下载新的依赖项)。 - 在需要时,可以使用诸如
go get foo@v1.2.3
、go get foo@master
、go get foo@e3702bed2
或直接编辑go.mod
等命令选择更具体的依赖项版本。
简要介绍您可能使用的其他常见功能:
*go list-m all
- 查看构建中所有直接和间接依赖的最终版本(详细信息)
*go list-u-m all
- 查看所有的直接和间接依赖的可用次要版本和补丁升级(详细信息)
*go get-u
或'go get-u=patch` - 将所有直接和间接依赖更新为最新的次要版本或补丁升级(忽略预发布版本)(详细信息)
*go build./…
或'go test./…` - 从模块根目录运行时,构建或测试模块中的所有包(详细信息)
*go mod tidy
-从go.mod
删除不再需要的依赖,添加添加操作系统、体系结构或构建tags的所需的依赖项(详细信息)
*replace
或gohack
- 使用fork、本地副本或依赖的确切版本(详细信息)
*go mod vendor
- 可选步骤,创建vendor
目录(详细信息)
在阅读了接下来的四节“新概念”之后,您将获取足够的信息在大部分项目中开始使用Modules。查看上面的目录也很有用(包括常见问题的解答),以便熟悉更详细的主题列表。
这些部分提供了对主要新概念的高级介绍。欲了解更多细节和理由,请参阅这40分钟的介绍性视频Russ Cox描述设计背后的理念,官方提议文件,或更详细的起源vgo blog series。
module是作为一起进行版本化的相关go包的集合。 Modules记录精确的依赖性求并创建可复制的构建。
通常,版本控制库只包含存储库根目录中定义的一个模块。(一个存储库也支持多模块,但通常情况下,这会导致在单仓库单模块的基础上进行更多的工作)。
总结存储库、模块和包之间的关系:
*存储库包含一个或多个go模块。
*每个模块包含一个或多个GO包。
*每个包由单个目录中的一个或多个go源文件组成。
模块必须根据semver进行语义版本化,通常采用v(major)(minor)(patch)
的形式,如v0.1.0
、v1.2.3
或v1.5.0-rc.1
。前导v
是必需的。如果使用git,则发布tag并提交其版本。公共和私有模块存储库和代理正变得可用(请参阅下面的FAQ。
一个模块由go源文件树定义,其根目录中有一个go.mod
文件。
模块源代码可能位于GoPath之外。这里包含了四个指令:module
、require
、replace
、exclude
。
下面是定义www.greatytc.com/my/thing
模块的go.mod
文件示例:
module www.greatytc.com/my/thing
require (
www.greatytc.com/some/dependency v1.2.3
www.greatytc.com/another/dependency/v4 v4.0.0
)
模块通过module
指令在go.mod
中声明其标识,也提供了其_模块路径_。模块中所有包的导入路径公共享这个模块路径作为公共前缀。模块路径和从'go.mod'到包目录的相对路径合在一起决定了包的导入路径。
例如,如果要为存储库www.greatytc.com/my/repo
创建一个模块,该存储库将包含两个导入路径为www.greatytc.com/my/repo/foo
和www.greatytc.com/my/repo/bar
的包,则go.mod
文件中的第一行通常会将模块路径声明为module www.greatytc.com/my/repo
,相应的磁盘结构如下:
repo/
├── go.mod
├── bar
│ └── bar.go
└── foo
└── foo.go
在go源代码中,包是使用完整路径(包括模块路径)导入的。例如,如果一个模块在其go.mod
中声明其标识为module example.com/my/module
,则用户可以执行以下操作:
import "example.com/my/module/mypkg"
这将从example.com/my/module
模块导入mypkg
包。
exclude
和replace
指令仅在当前模块(主模块)上起作用。在构建主模块时,主模块以外的模块中的exclude
和replace
指令将被忽略。因此,replace
和exclude
语句允许主模块完全控制自己的构建,而不受依赖项的完全控制。(参见 下面的FAQ 了解何时使用replace
指令)。
如果你在源代码中添加了一个新的导入,而这个导入还没有在go.mod
中的require
中覆盖,那么大多数go命令如go build
和go test
会自动查找正确的模块并添加 最高 版本的新直接依赖到你的模块的go.mod
的require
指令下面。 例如,如果新导入的最新标记发布版本为v1.2.3
的依赖项M,则模块的go.mod
将以require M v1.2.3
结束,这表示模块M是允许版本 >= v1.2.3的依赖(同时< v2,v2被认为与v1不兼容)。
最小版本选择(minimal version selection)算法用于选择构建中使用的所有模块的版本。对于构建中的每个模块,通过最小版本选择而选择的版本始终是主模块中的require
指令或其依赖项之一明确列出的版本的语义化最高。
例如,如果你的模块依赖于具有require D v1.0.0
的模块A,并且你的模块也依赖于具有require D v1.1.1
的模块B,那么最小版本选择将选择v1.1.1
的D包含在构建中(鉴于它是列出的最高require
版本)。 这种v1.1.1
的选择保持一致,即使一段时间后D的v1.2.0
变得可用。 这是模块系统如何提供100%可重复构建的示例。一切就绪后,模块作者或用户可能会选择升级到最新的D版本或选择D的显式版本。
有关最小版本选择算法的简要原理和概述,请参阅官方提供的高保真构建(High Fidelity Builds)部分,或参见更详细的vgo
博客系列。
要查看所选模块版本的列表(包括间接依赖),请使用go list -m all
。
参阅下面的 "How to Upgrade and Downgrade Dependencies"部分,以及FAQ "How are versions marked as incompatible?" 。
多年来,Go官方的FAQ 包含了关于包版本化的建议:
"供公众使用的软件包应该尝试在它们发展时保持向前兼容性。Go 1兼容性指南在这里是一个很好的参考:不要删除导出的名称,鼓励标记的复合文字等。如果需要不同的功能,添加新名称而不是更改旧名称。如果需要完整决裂,请创建一个带有新导入路径的新包。"
最后一句特别重要 —— 如果破坏了兼容性,则应更改包的导入路径。Go 1.11模块中,该建议被定义_import兼容性规则_:
"如果旧包和新包具有相同的导入路径, 新包必须向前兼容旧包。"
回想semver 在v1或更高版本的软件包进行向前不兼容的更改时更改主要版本。遵循导入兼容性规则和semver的结果称为_Semantic Import Versioning_,其中主要版本包含在导入路径中 —— 这可确保导入路径在主要版本因兼容性中断而增加时发生变化。
作为语义导入版本控制的结果,选择Go模块的代码必须符合这些规则:
- 遵循semver。 (例如 VCS 的tag是
v1.2.3
)。 - 如果模块版本为v2或更高版本,则模块的主要版本_必须_将
/vN
包含在go.mod
文件中使用的模块路径的末尾(例如,module www.greatytc.com/my/mod/v2
,require www.greatytc.com/my/mod/v2 v2.0.0
)和包导入路径(例如,import "www.greatytc.com/my/mod/v2/mypkg"
)。 - 如果模块是版本v0或v1,请_不要_在模块路径或导入路径中包含主要版本。
通常,具有不同导入路径的包是不同的包。例如,math/rand
与crypto/rand
是不同的包。如果导入路径中出现不同的主要版本导致不同的导入路径,则也是如此。因此example.com/my/mod/mypkg
与example.com/my/mod/v2/mypkg
是不同的包,并且两者都可以在一个构建中导入,这有助于解决金字塔依赖问题,并且还允许v1模块在其v2替换方面实现,反之亦然。
有关语义导入版本控制的更多详细信息,请参阅go
命令文档的"Module compatibility and semantic versioning"部分,并参阅https://semver.org 了解有关语义版本控制的更多信息。
到目前为止,本节主要关注选择模块和从其他模块导入代码。但是,将主要版本放在v2+模块的导入路径中可能与旧版本的Go或未使用模块的代码不兼容。为了解决这个问题,上述行为和规则有三个重要的过渡特例或例外。随着越来越多的软件包选择使用模块,这些过渡异常将变得越来越不重要。
三个过渡性例外
-
gopkg.in
甚至在选择使用模块后,使用以
gopkg.in
开头的导入路径的现有代码(例如gopkg.in/yaml.v1
和gopkg.in/yaml.v2
)可以继续将这些形式用于其模块路径和导入路径。 -
当导入非模块v2+包时'+incompatible'
模块可以导入未使用模块的v2+包。具有有效v2+ semvertag的非模块v2+包将在导入模块的
go.mod
文件中以+incompatible
后缀记录。+incompatible
后缀表示即使v2+包有一个有效的v2+ semvertag,如v2.0.0
,v2+包也没有主动使用模块,因此假设还_没有_创建v2+包,并且了解语义导入版本控制的含义以及如何在导入路径中使用主要版本。因此,在模块模式中操作时,go
工具会将非模块v2+包视为包的v1版本系列的(不兼容)扩展,并假设包不清楚语义导入版本控制,并且+incompatible
后缀表示go
工具也是这样做的。 -
未开启模块模式的"最小模块兼容性"
为了帮助实现向前兼容,Go版本1.9.7+,1.10.3+和1.11已经更新,以便使使用这些版本构建的代码更容易正确使用v2+模块_而不_需要修改现有代码。此行为称为"最小模块兼容性",它仅在完全模块模式 被禁用时生效,例如你在Go 1.11中设置了
GO111MODULE=off
,或者使用Go版本1.9.7+或1.10.3+。当依赖于Go 1.9.7+,1.10.3+和1.11中的这种"最小模块兼容性"机制时,_还未_选择使用模块的软件包将_不会_包含任何导入的v2+模块的导入路径中的主要版本。相比之下,_已经_使用模块的包_必须_包含导入路径中的主要版本以导入任何v2+模块(为了在go
工具以完全模块模式运行时正确导入v2+模块,并充分了解语义导入版本控制)。
有关发布v2+模块所需的确切机制,请参阅下述的"Releasing Modules (v2 or Higher)" 部分。
使用模块,有两个安装选项:
- 安装最新的Go 1.11版本。
-
从源码安装Go工具链
master
分支。
安装后,可以通过以下两种方式之一激活模块支持:
- 在
$GOPATH/src
树外部的目录中调用go
命令,当前目录或其任何父目录中有有效的go.mod
文件,并且未设置环境变量GO111MODULE
(或显式设置为auto
)。 - 调用
go
命令时使用GO111MODULE=on
环境变量集。
为现有项目创建go.mod
:
-
定位到GOPATH外部模块源树的根目录:
$ cd <project path outside $GOPATH/src> # e.g., cd ~/projects/hello
请注意,在GOPATH之外,不需要设置
GO111MODULE
来激活模块模式。 或者,想在自己的GOPATH中起作用:$ export GO111MODULE=on # 手动激活模块模式 $ cd $GOPATH/src/<project path> # e.g., cd $GOPATH/src/you/hello
-
创建初始模块定义并将其写入
go.mod
文件:$ go mod init
这个步骤从任何现有的
dep
Gopkg.lock
文件或者其他共支持九种依赖关系格式 , 添加依赖语句以匹配现有配置。go mod init
通常可以使用辅助数据(如VCS元数据)自动确定适当的模块路径,但如果go mod init
声明它不能自动确定模块路径,或者如果需要覆盖该路径,则可以提供模块路径作为go mod init
的可选参数,例如:$ go mod init www.greatytc.com/my/repo
请注意,如果依赖项包括v2+模块,或者如果你正在初始化v2+模块,那么在运行
go mod init
之后,可能还需要编辑go.mod
和.go
代码以添加/vN
以导入路径和模块路径,如"语义导入版本控制"部分所述。即使go mod init
自动从dep
或其他依赖关系管理转换了依赖关系信息,这同样适用。(因此,在运行go mod init
之后,通常不应该运行go mod tidy
,直到运行go build ./...
或类似的程序成功,这也是本节中显示的顺序)。 -
构建模块。当执行来自一个模块的根目录时,
./...
模式匹配所有在当前模块内的包。go build
将根据需要自动添加缺少的或未转换的依赖项,以满足特定构建调用的导入:$ go build ./...
-
按配置测试模块,以确保其与所选版本匹配:
$ go test ./...
-
(可选)运行模块测试以及所有直接和间接依赖项的测试,以检查不兼容性:
$ go test all
在标记发布之前,请参见下面章节"How to Prepare for a Release".
有关所有这些主题的更多信息,官方模块文档的主要入口点是golang.org提供.
应该使用go get
来完成日常升级和降级依赖项,这将自动更新go.mod
文件。 或者,你也可以直接编辑go.mod
。
此外,诸如'go build','go test'或甚至'go list'之类的命令将根据需要自动添加新的依赖项以满足导入(更新go.mod
并下载新的依赖项)。
要查看所有直接和间接依赖项的可用次要和补丁升级,请运行go list -u -m all
。
更新当前模块的所有直接和间接依赖到最新版本:
- 运行
go get -u
去使用最新的次要或补丁版本 - 运行
go get -u=patch
去使用最新补丁版本
go get foo
更新到最新版本的foo
。 go get foo
相当于go get foo @latest
—— 换句话说,如果没有指定@
版本,@latest
是默认值。
在本节中,"latest"是带有semver标记的最新版本,如果没有semver标记,则是最新的已知commit提交。 除非存储库中没有其他semver标记,否则预发布标记不会被选为"最新"(详细信息)。
一个常见的错误是认为go get -u foo
只能获得foo
的最新版本。 实际上,go get -u foo
或go get -u foo@latest
中的-u
意味着_也可以_获取foo
的直接和间接依赖_所有_的最新版本。 升级foo
的一个常见起点是运行go get foo
或go get foo@latest
而不用-u
(一切正常后,再考虑go get -u=patch foo
,go get -u=patch
,go get -u foo
或go get -u
)。
要升级或降级到更具体的版本,'go get'允许通过添加@version后缀或"模块查询" 来覆盖版本选择的包参数,例如go get foo @v1.6.2
,go get foo@e3702bed2
,或go get foo @'<v1.6.2'
。
使用诸如go get foo@master
之类的分支名称是获取最新提交的一种方式,无论它是否具有semver标记。
通常,不解析为semver标记的模块查询将在go.mod
文件中记录为伪版本。
请参阅"模块感知go get"和"模块查询"中go
命令文档的部分,以获取有关这些主题的更多信息。
模块能够使用尚未引入模块的软件包,包括在go.mod
中记录任何可用的semver标签,并使用这些semver标签进行升级或降级。 模块也可以使用没有任何正确的semver标签的软件包(在这种情况下,它们将使用go.mod
中的伪版本进行记录)。
在升级或降级任何依赖项之后,你可能希望为构建(包括直接和间接依赖)的所有包运行测试以检查不兼容性:
$ go test all
创建模块发布的最佳实践预计将作为初始模块实验的一部分出现。其中许多最终可能会是自动化未来的'go release'工具。
在标记发布之前要考虑的一些当前建议的最佳实践:
-
运行
go mod tidy
删除任何多余的依赖 (如这里所示),同时确保当前的go.mod反映了所有可能的构建tag/OS/架构组合(如这里所示)。- 相比之下,
go build
和go test
等其他命令不会从go.mod
中删除不再需要的依赖项,只会根据当前构建调用的tags/OS/架构更新go.mod
。
- 相比之下,
-
运行go test all`测试模块(包括运行直接和间接依赖的测试),以验证当前选定的包版本是否兼容。
- 可能的版本组合的数量在模块数量上是指数级的,因此一般来说,您不能期望依赖已经针对它们的依赖的所有可能组合进行了测试。
- 作为模块工作的一部分,
go test all
重新定义为更有用对于当前模块包含的所有包,以及它们通过一个或多个依赖的导入的所有包,同时排除在当前模块不相关的包。
-
确保
go.sum
随着go.mod
一起提交. 参见下述FAQ获取更过信息。
如果要发布v2或更高版本的模块,请首先查看上面的"语义导入版本控制"部分中的讨论,其中包括为什么主要版本包含在v2+模块的模块路径和导入路径中。同时,Go版本1.9.7+和1.10.3+已经更新以简化转换。
请注意,如果在采用模块之前,第一次为已经存在的仓库或一组已标记为v2.0.0
或更高版本的包采用模块,则建议的最佳实践将升级第一次采用模块时的主要版本。例如,如果您是foo
的作者,而foo
仓库的最新标记是v2.2.2
,而foo
尚未采用模块,则最佳做法是将v3.0.0
用于第一个采用模块的foo
版本的foo
(因此,第一个包含go.mod
文件的foo
版本)。在这种情况下,升级主版本可以为foo
的使用者提供更清晰的信息,如果需要,允许在foo
的v2系列上附加非模块补丁或次要版本,并为foo
的基于模块的使用者提供一个强大的信号,如果import "foo"
和相应的require foo v2.2.2+incompatible
,与import "foo/v3"
和相应的require foo/v3 v3.0.0
。(请注意,有关首次采用模块时升级主要版本的建议不适用于最新版本为v0.x.x或v1.x.x的包)。
发布v2或更高版本模块有两种可选机制。请注意,使用这两种技术,当模块作者推送新标签时,新的模块版本对消费者是可用的。使用创建v3.0.0
版本的示例,有两个选项:
-
主要分支: 更新
go.mod
文件,在在module
指令模块路径的末尾包含/v3
(例如module www.greatytc.com/my/module/v3
)。同时使用/v3
更新导入更新模块中的import语句(例如import "www.greatytc.com/my/module/v3/mypkg"
)。给发布打标签为v3.0.0
。- Go版本1.9.7+、1.10.3+和1.11能够正确使用此方法创建的v2+模块,而无需更新尚未选择使用模块的使用者代码(如"语义导入版本控制"中所述)。
- 社区工具www.greatytc.com/marwan-at-work/mod帮助自动执行此过程。参见repository 或者community tooling FAQ。
- 为了避免与此方法混淆,请考虑将模块的
v3.*.*
提交放在单独的v3分支上。 -
注意: 创建新分支_不是_必须的。如果您以前在master上发布过,并且希望在master上标记
v3.0.0
,那么这是一个可行的选项。(但是,请注意,在master
中引入不兼容的API更改可能会导致使用go get -u
而不清楚semver的非模块用户出现问题,因为go
具在go 1.11之前或模块模式在go 1.11+中未启用。 - 现有的依赖关系管理解决方案(如
dep
)目前在使用以这种方式创建的v2+模块时可能会遇到问题。请参见dep#1962。
-
主要子目录: 创建一个新的
v3
子目录 (例如my/module/v3
),并且放一个go.mod
文件。 模块路径必须以/v3
结尾. 拷贝或移动代码到v3
子目录。用/v3
更新模块中import语句 (例如import "www.greatytc.com/my/module/v3/mypkg"
)。给发布打标签为v3.0.0
。- 这提供了更大的向前兼容性。特别是,早于1.9.7和1.10.3的Go版本也能够正确地使用和构建使用这种方法创建的v2+模块。
- 这里更复杂的方法可以利用类型别名(在Go1.9中引入)并在位于不同子目录的主要版本之间传递。这可以提供额外的兼容性,并允许根据另一个主要版本实现一个主要版本,但对于模块作者来说需要做更多的工作。一个正在进行的自动化工具是
goforward
。请参阅此处了解更多详细信息和基本原理,以及goforward
的初始版本。 - 以前存在的依赖关系管理解决方案(如
dep
)应该能够使用以这种方式创建的v2+模块。
有关这些备选方案的更深入讨论,请参阅https://research.swtch.com/vgo-module。
可以通过将标签推送到包含模块源代码的仓库来发布新模块版本。标签是通过连接两个字符串形成的:前缀和版本。
版本是该版本的语义导入版本。 应该遵循以下语义化导入版本规则来选择。
前缀表示在仓库中定义模块的位置。 如果模块在仓库的根目录中定义,则前缀为空,标签只是版本。 但是,在多模块仓库中,前缀区分不同模块的版本。前缀是仓库中定义模块的目录。 如果仓库遵循上述主要子目录模式,则前缀不包括主要版本后缀。
例如,假设我们有一个模块 example.com/repo/sub/v2
,我们想要发布版本v2.1.6
。 仓库根对应于example.com/repo
,模块在仓库中的sub/v2/go.mod
中定义。该模块的前缀是sub/
。 此版本的完整标签应为sub/v2.1.6
。
本节试图简要列举迁移到模块时要做出的主要决策,以及列出其他与迁移相关的主题。 有关详细信息,请参考其他部分。
这些材料主要基于作为模块实验的一部分从社区中出现的最佳实践; 因此,这是一个正在进行中的部分,随着社区获得更多经验,该部分将随着时间的推移而改进。
摘要:
- 模块系统旨在允许整个Go生态系统中的不同包可以不同的速率进行引入。
- 已经在v2或更高版本上的软件包具有更多迁移注意事项,主要是由于语义导入版本控制的原因。
- 采用模块时,v0或v1版本以及新软件包的考虑因素要少得多。
- 使用Go 1.11定义的模块可以由较旧的Go版本使用(尽管确切的Go版本取决于主模块及其依赖关系所使用的策略,如下所述)。
迁移主题:
-
go mod init
自动化从dep, glide, govendor, godep和其他5个先前存在的依赖管理 转换所需的信息到可以产生一致性构建的go.mod
文件。 - 如果创建v2+模块,确保转换后的
go.mod
中的module
指令包含相应的/vN
(例如module foo/v3
)。 - 注意,如果要导入v2+模块,则可能需要在初始转换后进行一些手动调整,以便将
/vN
添加到之前的依赖管理转换后go mod init
生成的require
语句后。有关详细信息,请参阅上面的"如何定义模块"部分。 - 另外,
go mod init
不会编辑你的.go
代码来为import语句添加任何必需的/vN
。 请参阅"语义导入版本控制" 和"模块发布 (v2或更版本)"章节,介绍了所需的步骤,包括围绕自动转换的的社区工具一些选项。
- 较早版本的Go了解如何使用
go mod vendor
创建的vendor目录,当禁用模块模式时,Go 1.11和1.12+也是如此。因此,vendoring是模块为不完全理解模块的旧版Go提供依赖关系的一种方式,也为未启用模块的消费者提供了一种方式。 请参阅vendoring FAQ和go
命令文档了解更多详情。
- 在模块前,安装说明通常包含
go get -u foo
。 如果要发布模块foo
,请考虑在针对基于模块的使用者的说明中删除-u
。-
-u
要求go
工具升级foo
的所有直接和间接依赖关系。 - 模块使用者可以选择稍后再运行
go get -u foo
,但是"High Fidelity Builds"这样做有更多好处,如果`-u'不是初始安装说明的一部分。 详情参见"How to Upgrade and Downgrade Dependencies"。 -
go get -u foo
仍然有效,并且仍然可以作为安装说明的有效选择。
-
- 另外,对于基于模块的使用者来说,并不一定需要
go get foo
。- 只需添加一个导入语句
import "foo"
就足够了。 (后续的命令,例如go build或go test将自动下载foo并根据需要更新go.mod)。
- 只需添加一个导入语句
- 默认情况下,基于模块的使用者将不使用
vendor
目录。- 如果在
go
工具中启用了模块模式,则在使用模块时(严格根据go.mod
中包含的信息和go.sum
中的加密校验和进行比较),vendor
是非必须的,但某些现有的安装说明假定go
工具默认情况下将使用vendor
。详见vendoring FAQ。
- 如果在
- 在某些情况下,安装包含
go get foo/...
的说明可能会出现问题(详见#27215中的讨论)。
模块通过module
指令(例如module www.greatytc.com/my/module
)在其go.mod中声明其身份。 任何模块支持的使用者都必须使用与模块声明的模块路径匹配的导入路径(确切地说是针对根软件包,或将模块路径作为导入路径的前缀)导入模块内的所有软件包。如果导入路径与相应模块的声明模块路径不匹配,那么go
命令将报告unexpected module path
错误。
在为一组预先存在的软件包采用模块时,应注意避免破坏现有使用者使用的现有导入路径,除非在采用模块时增加主版本。
例如,如果之前存在的README文件一直在告诉消费者使用import "gopkg.in/foo.v1"
,然后如果采用模块到v1版本,那么最初的go.mod
应该几乎可以读为 module gopkg.in/foo.v1
。 如果您想放弃使用gopkg.in
,那对当前的消费者来说将是一个巨大的改变。 一种方法是,如果您后来转到v2,则更改为类似module www.greatytc.com/repo/foo/v2
之类的东西。
请注意,模块路径和导入路径区分大小写。例如,将模块从www.greatytc.com/Sirupsen/logrus
更改为www.greatytc.com/sirupsen/logrus
对消费者来说是重大更改,即使GitHub自动从一个存储库名称转发到新的存储库名称。
在采用模块之后,更改go.mod
中的模块路径是一项重大更改。
总的来说,这类似于通过 "import path comments"对规范导入路径的模块前强制实施,有时也称为"import pragmas"或"import path enforcement"。 例如,软件包go.uber.org/zap
当前托管在www.greatytc.com/uber-go/zap
,但是使用导入路径注释在软件包声明旁边对于基于github的导入路径如package zap // import "go.uber.org/zap"
的预模块使用者会触发一个错误。
go.mod文件的module语句已淘汰了导入路径注释。
- 如果您在采用模块之前已将其软件包标记为V2.0.0或更高版本,则建议的最佳实践是在首次采用模块时增加主要版本。例如,如果您使用的是
v2.0.1
版本,但尚未采用模块,则对于采用模块的第一个发行版,应使用v3.0.0
版本。查看 "Releasing Modules (v2 or Higher)" 章节获取更多信息。
- 如果某个模块在V2或更高版本上,则意味着多个主要版本可以位于一个内部版本中(例如,
foo
和foo/v3
可能会在单个内部版本中结束)。- 这自然源于"具有不同导入路径的包是不同的包"的规则。
- 发生这种情况时,将有多个软件包级别状态的副本(例如,
foo
的软件包级别状态和foo/v3
的软件包级别状态),并且每个主要版本都将运行其自己的init
方法。 - 这种方法有助于模块系统的多个方面,包括帮助解决钻石依赖问题,在大型代码库中逐步迁移到新版本,以及允许将主要版本实现为围绕其他主要版本的补充。
- 查看https://research.swtch.com/vgo-import的"Avoiding Singleton Problems"或#27514获取相关讨论。
- 模块能够使用尚未采用模块的软件包,并在导入模块的
go.mod
中记录适当的软件包版本信息。模块可以使用尚没有适当的semver标签的软件包。 有关更多信息,请参见FAQ below获取更多解答。 - 模块也可以导入未选择模块的v2+软件包。如果导入的v2+软件包具有有效的semver标签,则将以
+incompatible
后缀记录。请参见FAQ below 获取更多细节。
-
非模块代码使用v0和v1模块:
- 尚未选择使用模块的代码可以使用和构建v0和v1模块(与使用的Go版本无关)。
-
非模块代码使用v2+模块:
-
Go版本1.9.7+,1.10.3+和1.11已更新,因此使用这些发行版构建的代码可以正确使用v2+模块,而无需按照上述章节"Semantic Import versioning"和"Releasing Modules (v2 or Higher)"中所述修改现有代码。
-
如果按照"Releasing Modules (v2 or Higher)"中概述的"主要子目录"方法创建了v2+模块,则1.9.7和1.10.3之前的Go版本可以使用v2+模块。
-
对于考虑使用模块的预先存在的v2+软件包的作者,总结替代方法的一种方法是在三种顶级策略之间进行选择。每个选择都有后续的决定和变化(如上所述)。 这些替代的顶级策略是:
-
要求客户端使用Go版本1.9.7+,1.10.3+或1.11+。
该方法使用"主要分支"方法,并依赖"最小模块感知",该感知被反向移植到1.9.7和1.10.3。 请参阅"Semantic Import versioning" 和"Releasing Modules (v2 or Higher)"章节获取更多细节。
-
允许客户端使用像1.8的更老Go版本.
该方法使用"主要子目录"方法,涉及创建子目录,例如
/v2
或/v3
。请参阅"Semantic Import versioning" 和"Releasing Modules (v2 or Higher)"章节获取更多细节。 -
等待使用模块.
在此策略中,使用模块的客户端代码以及未使用模块的客户端代码一起使用。随着时间的流逝,Go版本1.9.7+,1.10.3+和1.11+的发布时间将越来越长,并且在将来的某个时候,要求Go版本变得更加自然或对客户友好1.9.7+/1.10.3+/1.11 +,此时,您可以实施以上策略1(需要Go版本1.9.7+,1.10.3+或1.11+),甚至可以实施以上策略2(但是如果最终要采用上述策略2来支持1.8等旧版Go,那么您现在就可以这样做)。
- Official documentation:
- Latest HTML documentation for modules on golang.org
- Run
go help modules
for more about modules. (This is the main entry point for modules topics viago help
) - Run
go help mod
for more about thego mod
command. - Run
go help module-get
for more about the behavior ofgo get
when in module-aware mode. - Run
go help goproxy
for more about the module proxy, including a pure file-based option via afile:///
URL.
- The initial "Go & Versioning" series of blog posts on
vgo
by Russ Cox (first posted February 20, 2018) - Official golang.org blog post introducing the proposal (March 26, 2018)
- This provides a more succinct overview of the proposal than the full
vgo
blog series, along with some of the history and process behind the proposal
- This provides a more succinct overview of the proposal than the full
- Official Versioned Go Modules Proposal (last updated March 20, 2018)
- Introductory 40 minute video "The Principles of Versions in Go" from GopherCon Singapore by Russ Cox (May 2, 2018)
- Succinctly covers the philosophy behind the design of versioned Go modules, including the three core principles of "Compatibility", "Repeatability", and "Cooperation"
- Example based 35 minute introductory video "What are Go modules and how do I use them?" (slides) by Paul Jolly (August 15, 2018)
- Introductory blog post "Taking Go Modules for a Spin" by Dave Cheney (July 14, 2018)
- Introductory Go Meetup slides on modules by Chris Hines (July 16, 2018)
- Introductory 30 minute video "Intro to Go Modules and SemVer" by Francesc Campoy (Nov 15, 2018)
- Blog post "Using Go modules with vendor support on Travis CI" by Fatih Arslan (August 26, 2018)
- Blog post "Go Modules and CircleCI" by Todd Keech (July 30, 2018)
- Blog post "The vgo proposal is accepted. Now what?" by Russ Cox (May 29, 2018)
- Includes summary of what it means that versioned modules are currently an experimental opt-in feature
- Blog post on how to build go from tip and start using go modules by Carolyn Van Slyck (July 16, 2018)
As part of the proposal, prototype, and beta processes, there have been over 400 issues created by the overall community. Please continue to supply feedback.
Here is a partial list of some of the larger changes and improvements, almost all of which were primarily based on feedback from the community:
- Top-level vendor support was retained rather than vgo-based builds ignoring vendor directories entirely (discussion, CL)
- Backported minimal module-awareness to allow older Go versions 1.9.7+ and 1.10.3+ to more easily consume modules for v2+ projects (discussion, CL)
- Allowed vgo to use v2+ tags by default for pre-existing packages did not yet have a go.mod (recent update in related behavior described here)
- Added support via command
go get -u=patch
to update all transitive dependencies to the latest available patch-level versions on the same minor version (discussion, documentation) - Additional control via environmental variables (e.g., GOFLAGS in #26585, CL)
- Finer grain control on whether or not go.mod is allowed to be updated, how vendor directory is used, and whether or not network access is allowed (e.g., -mod=readonly, -mod=vendor, GOPROXY=off; related CL for recent change)
- Added more flexible replace directives (CL)
- Added additional ways to interrogate modules (for human consumption, as well as for better editor / IDE integration)
- The UX of the go CLI has continued to be refined based on experiences so far (e.g., #26581, CL)
- Additional support for warming caches for use cases such as CI or docker builds via
go mod download
(#26610) - Most likely: better support for installing specific versions of programs to GOBIN (#24250)
- Currently open module issues
- Closed module issues
- Closed vgo issues
- Submit a new module issue using 'cmd/go:' as the prefix
The require
directive allows any module to declare that it should be built with version >= x.y.z of a dependency D (which may be specified due to incompatibilities with version < x.y.z of module D). Empirical data suggests this is the dominant form of constraints used in dep
and cargo
. In addition, the top-level module in the build can exclude
specific versions of dependencies or replace
other modules with different code. See the full proposal for more details and rationale.
One of the key goals of the versioned modules proposal is to add a common vocabulary and semantics around versions of Go code for both tools and developers. This lays a foundation for future capabilities to declare additional forms of incompatibilities, such as possibly:
- declaring deprecated versions as described in the initial
vgo
blog series - declaring pair-wise incompatibility between modules in an external system as discussed for example here during the proposal process
- declaring pair-wise incompatible versions or insecure versions of a module after a release has been published. See for example the on-going discussion in #24031 and #26829
In general, modules are opt-in for Go 1.11, so by design old behavior is preserved by default.
Summarizing when you get the old 1.10 status quo behavior vs. the new opt-in modules-based behavior:
- Inside GOPATH — defaults to old 1.10 behavior (ignoring modules)
- Outside GOPATH while inside a file tree with a
go.mod
— defaults to modules behavior - GO111MODULE environment variable:
- unset or
auto
— default behavior above -
on
— force module support on regardless of directory location -
off
— force module support off regardless of directory location
- unset or
This occurs when you have set GO111MODULE=on
, but are not inside of a file tree with a go.mod
when you run go get
.
The simplest solution is to leave GO111MODULE
unset (or equivalently explicitly set to GO111MODULE=auto
), which avoids this error.
Recall one of the primary reason modules exist is to record precise dependency information. This dependency information is written to your current go.mod
. If you are not inside of a file tree with a go.mod
but you have told the go get
command to operate in module mode by setting GO111MODULE=on
, then running go get
will result in the error cannot find main module
because there is no go.mod
available to record dependency information.
Solution alternatives include:
-
Leave
GO111MODULE
unset (the default, or explicitly setGO111MODULE=auto
), which results in friendlier behavior. This will give you Go 1.10 behavior when you are outside of a module and hence will avoidgo get
reportingcannot find main module
. -
Leave
GO111MODULE=on
, but as needed disable modules temporarily and enable Go 1.10 behavior duringgo get
, such as viaGO111MODULE=off go get example.com/cmd
. This can be turned into a simple script or shell alias such asalias oldget='GO111MODULE=off go get'
-
Create a temporary
go.mod
file that is then discarded. This has been automated by a simple shell script by @rogpeppe. This script allows version information to optionally be supplied viavgoget example.com/cmd[@version]
. (This can be a solution for avoiding the errorcannot use path@version syntax in GOPATH mode
). -
gobin
is a module-aware command to install and run main packages. By default,gobin
installs/runs main packages without first needing to manually create a module, but with the-m
flag it can be told to use an existing module to resolve dependenci es. Please see thegobin
README and FAQ for details and additional use cases. -
Create a
go.mod
you use to track your globally installed tools, such as in~/global-tools/go.mod
, andcd
to that directory prior to runninggo get
orgo install
for any globally installed tools. -
Create a
go.mod
for each tool in separate directories, such as~/tools/gorename/go.mod
and~/tools/goimports/go.mod
, andcd
to that appropriate directory prior to runninggo get
orgo install
for the tool.
This current limitation will be resolved. However, the primary issue is that modules are currently opt-in, and a full solution will likely wait until GO111MODULE=on becomes the default behavior. See #24250 for more discussion, including this comment:
This clearly must work eventually. The thing I'm not sure about is exactly what this does as far as the version is concerned: does it create a temporary module root and go.mod, do the install, and then throw it away? Probably. But I'm not completely sure, and for now I didn't want to confuse people by making vgo do things outside go.mod trees. Certainly the eventual go command integration has to support this.
This FAQ has been discussing tracking globally installed tools.
If instead you want to track the tools required by a specific module, see the next FAQ.
If you:
- want to use a go-based tool (e.g.
stringer
) while working on a module, and - want to ensure that everyone is using the same version of that tool while tracking the tool's version in your module's
go.mod
file
then one currently recommended approach is to add a tools.go
file to your module that includes import statements for the tools of interest (such as import _ "golang.org/x/tools/cmd/stringer"
), along with a // +build tools
build constraint. The import statements allow the go
command to precisely record the version information for your tools in your module's go.mod
, while the // +build tools
build constraint prevents your normal builds from actually importing your tools.
For a concrete example of how to do this, please see this "Go Modules by Example" walkthrough.
A discussion of the approach along with an earlier concrete example of how to do this is in this comment in #25922.
The brief rationale (also from #25922):
I think the tools.go file is in fact the best practice for tool dependencies, certainly for Go 1.11.
I like it because it does not introduce new mechanisms.
It simply reuses existing ones.
What is the status of module support in IDEs, editors and standard tools like goimports, gorename, etc?
Support for modules is starting to land in editors and IDEs.
For example:
- GoLand: currently has full support for modules outside and inside GOPATH, including completion, syntax analysis, refactoring, navigation as described here.
- VS Code: work is in progress and looking for contributors to help. Tracking issue is #1532. An initial beta is described in the VS Code module status wiki page.
- Atom with go-plus: tracking issue is #761.
-
vim with vim-go: initial support for syntax highlighting and formatting
go.mod
has landed. Broader support tracked in #1906. - emacs with go-mode.el: tracking issue in #237.
The status of other tools such as goimports, guru, gorename and similar tools is being tracked in an umbrella issue #24661. Please see that umbrella issue for latest status.
Some tracking issues for particular tools includes:
-
gocode: tracking issue in mdempsky/gocode/#46. Note that
nsf/gocode
is recommending people migrate off ofnsf/gocode
tomdempsky/gocode
. - go-tools (tools by dominikh such as staticcheck, megacheck, gosimple): sample tracking issue dominikh/go-tools#328.
In general, even if your editor, IDE or other tools have not yet been made module aware, much of their functionality should work with modules if you are using modules inside GOPATH and do go mod vendor
(because then the proper dependencies should be picked up via GOPATH).
The full fix is to move programs that load packages off of go/build
and onto golang.org/x/tools/go/packages
, which understands how to locate packages in a module-aware manner. This will likely eventually become go/packages
.
The community is starting to build tooling on top of modules. For example:
-
www.greatytc.com/rogpeppe/gohack
- A new community tool to automate and greatly simplify
replace
and multi-module workflows, including allowing you to easily modify one of your dependencies - For example,
gohack example.com/some/dependency
automatically clones the appropriate repository and adds the necessaryreplace
directives to yourgo.mod
- Remove all gohack replace statements with
gohack undo
- The project is continuing to expand to make other module-related workflows easier
- A new community tool to automate and greatly simplify
-
www.greatytc.com/marwan-at-work/mod
- Command line tool to automatically upgrade/downgrade major versions for modules
- Automatically adjusts
go.mod
files and related import statements in go source code - Helps with upgrades, or when first opting in to modules with a v2+ package
-
www.greatytc.com/akyoto/mgit
- Lets you view & control semver tags of all of your local projects
- Shows untagged commits and lets you tag them all at once (
mgit -tag +0.0.1
)
-
www.greatytc.com/goware/modvendor
- Helps copy additional files into the
vendor/
folder, such as shell scripts, .cpp files, .proto files, etc.
- Helps copy additional files into the
As described in the 'go.mod' concepts section above, replace
directives provide additional control in the top-level go.mod
for what is actually used to satisfy a dependency found in the Go source or go.mod files, while replace
directives in modules other than the main module are ignored when building the main module.
The replace
directive allows you to supply another import path that might be another module located in VCS (GitHub or elsewhere), or on your local filesystem with a relative or absolute file path. The new import path from the replace
directive is used without needing to update the import paths in the actual source code.
replace
allows the top-level module control over the exact version used for a dependency, such as:
replace example.com/some/dependency => example.com/some/dependency v1.2.3
replace
also allows the use of a forked dependency, such as:
replace example.com/some/dependency => example.com/some/dependency-fork v1.2.3
One sample use case is if you need to fix or investigate something in a dependency, you can have a local fork and add the something like the following in your top-level go.mod
:
replace example.com/original/import/path => /your/forked/import/path
replace
also can be used to inform the go tooling of the relative or absolute on-disk location of modules in a multi-module project, such as:
replace example.com/project/foo => ../foo
Note: if the right-hand side of a replace
directive is a filesystem path, then the target must have a go.mod
file at that location. If the go.mod
file is not present, you can create one with go mod init
.
In general, you have the option of specifying a version to the left of the =>
in a replace directive, but typically it is less sensitive to change if you omit that (e.g., as done in all of the replace
examples above).
In Go 1.11, for direct dependencies a require
directive is needed even when doing a replace
. For example, if foo
is a direct dependency, you cannot do replace foo => ../foo
without a corresponding require
for foo
. If you are not sure what version to use in the require
directive, you can often use v0.0.0
such as require foo v0.0.0
. This was addressed in Go 1.12 with #26241.
You can confirm you are getting your expected versions by running go list -m all
, which shows you the actual final versions that will be used in your build including taking into account replace
statements.
See the 'go mod edit' documentation for more details.
www.greatytc.com/rogpeppe/gohack makes these types of workflows much easier, especially if your goal is to have mutable checkouts of dependencies of a module. See the repository or the immediately prior FAQ for an overview.
See the next FAQ for the details of using replace
to work entirely outside of VCS.
Yes. VCS is not required.
This is very simple if you have a single module you want to edit at a time outside of VCS (and you either have only one module in total, or if the other modules reside in VCS). In this case, you can place the file tree containing the single go.mod
in a convenient location. Your go build
, go test
and similar commands will work even if your single module is outside of VCS (without requiring any use of replace
in your go.mod
).
If you want to have multiple inter-related modules on your local disk that you want to edit at the same time, then replace
directives are one approach. Here is a sample go.mod
that uses a replace
with a relative path to point the hello
module at the on-disk location of the goodbye
module (without relying on any VCS):
module example.com/me/hello
require (
example.com/me/goodbye v0.0.0
)
replace example.com/me/goodbye => ../goodbye
As shown in this example, if outside of VCS you can use v0.0.0
as the version in the require
directive. Note that as mentioned in the prior FAQ, in Go 1.11 a require
directive must be manually added here, but that require
directive no longer needs to be manually added in Go 1.12+ (#26241).
A small runnable example is shown in this thread.
The initial series of vgo
blog posts did propose dropping vendoring entirely, but feedback from the community resulted in retaining support for vendoring.
In brief, to use vendoring with modules:
-
go mod vendor
resets the main module's vendor directory to include all packages needed to build and test all of the module's packages based on the state of the go.mod files and Go source code. - By default, go commands like
go build
ignore the vendor directory when in module mode. - The
-mod=vendor
flag (e.g.,go build -mod=vendor
) instructs the go commands to use the main module's top-level vendor directory to satisfy dependencies. The go commands in this mode therefore ignore the dependency descriptions in go.mod and assume that the vendor directory holds the correct copies of dependencies. Note that only the main module's top-level vendor directory is used; vendor directories in other locations are still ignored. - Some people will want to routinely opt-in to vendoring by setting a
GOFLAGS=-mod=vendor
environment variable.
Older versions of Go such as 1.10 understand how to consume a vendor directory created by go mod vendor
, as do Go 1.11 and 1.12+ when module mode is disabled. Therefore, vendoring is one way for a module to provide dependencies to older versions of Go that do not fully understand modules, as well as to consumers that have not enabled modules themselves.
If you are considering using vendoring, it is worthwhile to read the "Modules and vendoring" and "Make vendored copy of dependencies" sections of the tip documentation.
Publicly hosted "always on" immutable module repositories and optional privately hosted proxies and repositories are becoming available.
For example:
- Project Athens: Open source project in the works and looking for contributors.
- JFrog Artifactory: Commercial offering. Support for Go 1.11 modules started with release 5.11 as described here and here. From Artifactory version 6.2.0, please use JFrog CLI 1.20.2 and above.
- goproxy.io - A Global Proxy for Go Modules.
- THUMBAI - Open source project - Go Mod Proxy server and Go Vanity Import Path server
- Goproxy - Open source project - A minimalist Go module proxy handler.
- Goproxy China - Open source project - The most trusted Go module proxy in China.
Note that you are not required to run a proxy. Rather, the go tooling in 1.11 has added optional proxy support via GOPROXY to enable more enterprise use cases (such as greater control), and also to better handle situations such as "GitHub is down" or people deleting GitHub repositories.
Can I control when go.mod gets updated and when the go tools use the network to satisfy dependencies?
By default, a command like go build
will reach out to the network as needed to satisfy imports.
Some teams will want to disallow the go tooling from touching the network at certain points, or will want greater control regarding when the go tooling updates go.mod
, how dependencies are obtained, and how vendoring is used.
The go tooling provides a fair amount of flexibility to adjust or disable these default behaviors, including via -mod=readonly
, -mod=vendor
, GOFLAGS
, GOPROXY=off
, GOPROXY=file:///filesystem/path
, go mod vendor
, and go mod download
.
The details on these options are spread throughout the official documentation. One community attempt at a consolidated overview of knobs related to these behaviors is here, which includes links to the official documentation for more information.
The simplest approach is likely just setting the environment variable GO111MODULE=on
, which should work with most CI systems.
However, it can be valuable to run tests in CI on Go 1.11 with modules enabled as well as disabled, given some of your users will not have yet opted in to modules themselves. Vendoring is also a topic to consider.
The following two blog posts cover these topics more concretely:
- "Using Go modules with vendor support on Travis CI" by Fatih Arslan
- "Go Modules and CircleCI" by Todd Keech
The modules system records precise dependency requirements in your go.mod
. (For more details, see the go.mod concepts section above or the go.mod tip documentation).
go mod tidy
updates your current go.mod
to include the dependencies needed for tests in your module — if a test fails, we must know which dependencies were used in order to reproduce the failure.
go mod tidy
also ensures your current go.mod
reflects the dependency requirements for all possible combinations of OS, architecture, and build tags (as described here). In contrast, other commands like go build
and go test
only update go.mod
to provide the packages imported by the requested packages under the current GOOS
, GOARCH
, and build tags (which is one reason go mod tidy
might add requirements that were not added by go build
or similar).
If a dependency of your module does not itself have a go.mod
(e.g., because the dependency has not yet opted in to modules itself), or if its go.mod
file is missing one or more of its dependencies (e.g., because the module author did not run go mod tidy
), then the missing transitive dependencies will be added to your module's requirements, along with an // indirect
comment to indicate that the dependency is not from a direct import within your module.
Note that this also means that any missing test dependencies from your direct or indirect dependencies will also be recorded in your go.mod
. (An example of when this is important: go test all
runs the tests of all direct and indirect dependencies of your module, which is one way to validate that your current combination of versions work together. If a test fails in one of your dependencies when you run go test all
, it is important to have a complete set of test dependency information recorded so that you have reproducible go test all
behavior).
Another reason you might have // indirect
dependencies in your go.mod
file is if you have upgraded (or downgraded) one of your indirect dependencies beyond what is required by your direct dependencies, such as if you ran go get -u
or go get foo@1.2.3
. The go tooling needs a place to record those new versions, and it does so in your go.mod
file (and it does not reach down into your dependencies to modify their go.mod
files).
In general, the behaviors described above are part of how modules provide 100% reproducible builds and tests by recording precise dependency information.
If you are curious as to why a particular module is showing up in your go.mod
, you can run go mod why -m <module>
to answer that question. Other useful tools for inspecting requirements and versions include go mod graph
and go list -m all
.
Is 'go.sum' a lock file? Why does 'go.sum' include information for module versions I am no longer using?
No, go.sum
is not a lock file. The go.mod
files in a build provide enough information for 100% reproducible builds.
For validation purposes, go.sum
contains the expected cryptographic checksums of the content of specific module versions. See the FAQ below for more details on go.sum
(including why you typically should check in go.sum
) as well as the "Module downloading and verification" section in the tip documentation.
In part because go.sum
is not a lock file, it retains cryptographic checksums for module versions even after you stop using a module or particular module version. This allows validation of the checksums if you later resume using something, which provides additional safety.
In addition, your module's go.sum
records checksums for all direct and indirect dependencies used in a build (and hence your go.sum
will frequently have more modules listed than your go.mod
).
Typically your module's go.sum
file should be committed along with your go.mod
file.
-
go.sum
contains the expected cryptographic checksums of the content of specific module versions. - If someone clones your repository and downloads your dependencies using the go command, they will receive an error if there is any mismatch between their downloaded copies of your dependencies and the corresponding entries in your
go.sum
. - In addition,
go mod verify
checks that the on-disk cached copies of module downloads still match the entries ingo.sum
. - Note that
go.sum
is not a lock file as used in some alternative dependency management systems. (go.mod
provides enough information for reproducible builds). - See very brief rationale here from Filippo Valsorda on why you should check in your
go.sum
. See the "Module downloading and verification" section of the tip documentation for more details. See possible future extensions being discussed for example in #24117 and #25530.
Yes. This supports working outside of GOPATH, helps communicate to the ecosystem that you are opting in to modules, and in addition the module
directive in your go.mod
serves as a definitive declaration of the identity of your code (which is one reason why import comments might eventually be deprecated). Of course, modules are purely an opt-in capability in Go 1.11.
Please see the discussion on the Semantic Import Versioning and the import compatibility rule in the "Semantic Import Versioning" concepts section above. See also the blog post announcing the proposal, which talks more about the motivation and justification for the import compatibility rule.
Please see the question "Why are major versions v0, v1 omitted from import paths?" in the earlier FAQ from the official proposal discussion.
What are some implications of tagging my project with major version v0, v1, or making breaking changes with v2+?
In response to a comment about "k8s does minor releases but changes the Go API in each minor release", Russ Cox made the following response that highlights some implications for picking v0, v1, vs. frequently making breaking changes with v2, v3, v4, etc. with your project:
I don't fully understand the k8s dev cycle etc, but I think generally the k8s team needs to decide/confirm what they intend to guarantee to users about stability and then apply version numbers accordingly to express that.
- To make a promise about API compatibility (which seems like the best user experience!) then start doing that and use 1.X.Y.
- To have the flexibility to make backwards-incompatible changes in every release but allow different parts of a large program to upgrade their code on different schedules, meaning different parts can use different major versions of the API in one program, then use X.Y.0, along with import paths like k8s.io/client/vX/foo.
- To make no promises about API compatible and also require every build to have only one copy of the k8s libraries no matter what, with the implied forcing of all parts of a build to use the same version even if not all of them are ready for it, then use 0.X.Y.
On a related note, Kubernetes has some atypical build approaches (currently including custom wrapper scripts on top of godep), and hence Kubernetes is an imperfect example for many other projects, but it will likely be an interesting example as Kubernetes moves towards adopting Go 1.11 modules.
Yes.
If a repository has not opted in to modules but has been tagged with valid semver tags (including the required leading v
), then those semver tags can be used in a go get
, and a corresponding semver version will be record in the importing module's go.mod
file. If the repository does not have any valid semver tags, then the repository's version will be recorded with a "pseudo-version" such as v0.0.0-20171006230638-a6e239ea1c69
(which includes a timestamp and a commit hash, and which are designed to allow a total ordering across versions recored in go.mod
and to make it easier to reason about which recorded versions are "later" than another recorded version).
For example, if the latest version of package foo
is tagged v1.2.3
but foo
has not itself opted in to modules, then running go get foo
or go get foo@v1.2.3
from inside module M will be recorded in module M's go.mod
file as:
require foo v1.2.3
The go
tool will also use available semver tags for a non-module package in additional workflows (such as go list -u=patch
, which upgrades the dependencies of a module to available patch releases, or go list -u -m all
, which shows available upgrades, etc.).
Please see the next FAQs for additional details related to v2+ packages that have not opted in to modules.
Yes, a module can import a v2+ package that has not opted into modules, and if the imported v2+ package has a valid semver tag, it will be recorded with an +incompatible
suffix.
Additional Details
Please be familiar with the material in the "Semantic Import Versioning" section above.
It is helpful to first review some core principles that are generally useful but particularly important to keep in mind when thinking about the behavior described in this FAQ.
The following core principles are always true when the go
tool is operating in module mode (e.g., GO111MODULE=on
):
- A package's import path defines the identity of the package.
- Packages with different import paths are treated as different packages.
- Packages with the same import path are treated as the same package (and this is true even if the VCS tags say the packages have different major versions).
- An import path without a
/vN
is treated as a v1 or v0 module (and this is true even if the imported package has not opted in to modules and has VCS tags that say the major version is greater than 1). - The module path (such as
module foo/v2
) declared at the start of a module'sgo.mod
file is both:- the definitive declaration of that module's identity
- the definitive declaration of how that module must be imported by consuming code
As we will see in the next FAQ, these principles are not always true when the go
tool is not in module mode, but these principles are always true when the go
tool is in module mode.
In short, the +incompatible
suffix indicates that principle 2 above is in effect when the following are true:
- an imported package has not opted in to modules, and
- its VCS tags say the major version is greater than 1, and
- principle 2 is overriding the VCS tags – the import path without a
/vN
is treated as a v1 or v0 module (even though the VCS tags say otherwise)
When the go
tool is in module mode, it will assume a non-module v2+ package has no awareness of Semantic Import Versioning and treat it as an (incompatible) extension of the v1 version series of the package (and the +incompatible
suffix is an indication that the go
tool is doing so).
Example
Suppose:
-
oldpackage
is a package that predates the introduction of modules -
oldpackage
has never opted in to modules (and hence does not have ago.mod
itself) -
oldpackage
has a valid semver tagv3.0.1
, which is its latest tag
In this case, running for example go get oldpackage@latest
from inside module M will record the following in module M's go.mod
file:
require oldpackage v3.0.1+incompatible
Note that there is no /v3
used at the end of oldpackage
in the go get
command above or in the recorded require
directive – using /vN
in module paths and import paths is a feature of Semantic Import Versioning, and oldpackage
has not signaled its acceptance and understanding of Semantic Import Versioning given oldpackage
has not opted into modules by having a go.mod
file within oldpackage
itself. In other words, even though oldpackage
has a semver tag of v3.0.1
, oldpackage
is not granted the rights and responsibilities of Semantic Import Versioning (such as using /vN
in import paths) because oldpackage
has not yet stated its desire to do so.
The +incompatible
suffix indicates that the v3.0.1
version of oldpackage
has not actively opted in to modules, and hence the v3.0.1
version of oldpackage
is assumed to not understand Semantic Import Versioning or how to use major versions in import paths. Therefore, when operating in module mode, the go
tool will treat the non-module v3.0.1
version of oldpackage
as an (incompatible) extension of the v1 version series of oldpackage
and assume that the v3.0.1
version of oldpackage
has no awareness of Semantic Import Versioning, and the +incompatible
suffix is an indication that the go
tool is doing so.
The fact that the the v3.0.1
version of oldpackage
is considered to be part of the v1 release series according to Semantic Import Versioning means for example that versions v1.0.0
, v2.0.0
, and v3.0.1
are all always imported using the same import path:
import "oldpackage"
Note again that there is no /v3
used at the end of oldpackage
.
In general, packages with different import paths are different packages. In this example, given versions v1.0.0
, v2.0.0
, and v3.0.1
of oldpackage
would all be imported using the same import path, they are therefore treated by a build as the same package (again because oldpackage
has not yet opted in to Semantic Import Versioning), with a single copy of oldpackage
ending up in any given build. (The version used will be the semantically highest of the versions listed in any require
directives; see "Version Selection").
If we suppose that later a new v4.0.0
release of oldpackage
is created that adopts modules and hence contains a go.mod
file, that is the signal that oldpackage
now understands the rights and responsibilities of Semantic Import Versioning, and hence a module-based consumer would now import using /v4
in the import path:
import "oldpackage/v4"
and the version would be recorded as:
require oldpackage/v4 v4.0.0
oldpackage/v4
is now a different import path than oldpackage
, and hence a different package. Two copies (one for each import path) would end up in a module-aware build if some consumers in the build have import "oldpackage/v4"
while other consumers in the same build have import "oldpackage"
. This is desirable as part of the strategy to allow gradual adoption of modules. In addition, even after modules are out of their current transitional phase, this behavior is also desirable to allow gradual code evolution over time with different consumers upgrading at different rates to newer versions (e.g., allowing different consumers in a large build to choose to upgrade at different rates from oldpackage/v4
to some future oldpackage/v5
).
How are v2+ modules treated in a build if modules support is not enabled? How does "minimal module compatibility" work in 1.9.7+, 1.10.3+, and 1.11?
When considering older Go versions or Go code that has not yet opted in to modules, Semantic Import Versioning has significant backwards compatibility implications related to v2+ modules.
As described in the "Semantic Import Versioning" section above:
- a module that is version v2 or higher must include a
/vN
in its own module path declared in itsgo.mod
. - a module-based consumer (that is, code that has opted in to modules) must include a
/vN
in the import path to import a v2+ module.
However, the ecosystem is expected to proceed at varying paces of adoption for modules and Semantic Import Versioning.
As described in more detail in the "How to Release a v2+ Module" section, in the "Major Subdirectory" approach, the author of a v2+ module creates subdirectories such as mymodule/v2
or mymodule/v3
and moves or copies the approriate packages underneath those subdirectories. This means the traditional import path logic (even in older Go releases such as Go 1.8 or 1.7) will find the appropriate packages upon seeing an import statement such as import "mymodule/v2/mypkg"
. Hence, packages residing in a "Major Subdirectory" v2+ module will be found and used even if modules support is not enabled (whether that is because you are running Go 1.11 and have not enabled modules, or because you are running a older version like Go 1.7, 1.8, 1.9 or 1.10 that does not have full module support). Please see the "How to Release a v2+ Module" section for more details on the "Major Subdirectory" approach.
The remainder of this FAQ is focused on the "Major Branch" approach described in the "How to Release a v2+ Module" section. In the "Major Branch" approach, no /vN
subdirectories are created and instead the module version information is communicated by the go.mod
file and by applying semver tags to commits (which often will be on master
, but could be on different branches).
In order to help during the current transitional period, "minimal module compatibility" was introduced to Go 1.11 to provide greater compatibility for Go code that has not yet opted in to modules, and that "minimal module compatibility" was also backported to Go 1.9.7 and 1.10.3 (where those versions are effectively always operating with full module mode disabled given those older Go versions do not have full module support).
The primary goals of "minimal module compatibility" are:
-
Allow older Go versions 1.9.7+ and 1.10.3+ to be able to more easily compile modules that are using Semantic Import Versioning with
/vN
in import paths, and provide that same behavior when module mode is disabled in Go 1.11. -
Allow old code to be able to consume a v2+ module without requiring that old consumer code to immediately change to using a new
/vN
import path when consuming a v2+ module. -
Do so without relying on the module author to create
/vN
subdirectories.
Additional Details – "Minimal Module Compatibility"
"Minimal module compatibility" only takes effect when full module mode is disabled for the go
tool, such as if you have set GO111MODULE=off
in Go 1.11, or are using Go versions 1.9.7+ or 1.10.3+.
When a v2+ module author has not created /v2
or /vN
subdirectories and you are instead relying on the "minimal module compatibility" mechanism in Go 1.9.7+, 1.10.3+ and 1.11:
- A package that has not opted in to modules would not include the major version in the import path for any imported v2+ modules.
- In contrast, a package that has opted in to modules must include the major version in the import path to import any v2+ modules.
- If a package has opted in to modules, but does not include the major version in the import path when importing a v2+ modules, it will not import a v2+ version of that module when the
go
tool is operating in full module mode. (A package that has opted in to modules is assumed to "speak" Semantic Import Versioning. Iffoo
is a module with v2+ versions, then under Semantic Import Versioning sayingimport "foo"
means import the v1 Semantic Import Versioning series offoo
).
- If a package has opted in to modules, but does not include the major version in the import path when importing a v2+ modules, it will not import a v2+ version of that module when the
- The mechanism used to implement "minimal module compatibility" is intentionally very narrow:
- The entirety of the logic is – when operating in GOPATH mode, an unresolvable import statement containing a
/vN
will be tried again after removing the/vN
if the import statement is inside code that has opted in to modules (that is, import statements in.go
files within a tree with a validgo.mod
file). - The net effect is that an import statement such as
import "foo/v2"
within code that lives inside of a module will still compile correctly in GOPATH mode in 1.9.7+, 1.10.3+ and 1.11, and it will resolve as if it saidimport "foo"
(without the/v2
), which means it will use the version offoo
that resides in your GOPATH without being confused by the extra/v2
. - "Minimal module compatibility" does not affect anything else, including it does not the affect paths used in the
go
command line (such as arguments togo get
orgo list
).
- The entirety of the logic is – when operating in GOPATH mode, an unresolvable import statement containing a
- This transitional "minimal module awareness" mechanism purposefully breaks the rule of "packages with different import paths are treated as different packages" in pursuit a very specific backwards compatibility goal – to allow old code to compile unmodified when it is consuming a v2+ module. In slightly more detail:
- It would be a more burdensome for the overall ecosystem if the only way for old code to consume a v2+ module was to first change the old code.
- If we are not modifying old code, then that old code must work with pre-module import paths for v2+ modules.
- On the other hand, new or updated code opting in to modules must use the new
/vN
import for v2+ modules. - The new import path is not equal to old import path, yet both are allowed to work in a single build, and therefore we have two different functioning import paths that resolve to the same package.
- For example, when operating in GOPATH mode,
import "foo/v2"
appearing in module-based code resolves to the same code residing in your GOPATH asimport "foo"
, and the build ends up with one copy offoo
– in particular, whatever version is on disk in GOPATH. This allows module-based code withimport "foo/v2"
to compile even in GOPATH mode in 1.9.7+, 1.10.3+ and 1.11.
- In contrast, when the
go
tool is operating in full module mode:- There are no exceptions to the rule "packages with different import paths are different packages" (including vendoring has been refined in full module mode to also adhere to this rule).
- For example, if the
go
tool is in full module mode andfoo
is a v2+ module, thenimport "foo"
is asking for a v1 version offoo
vs.import "foo/v2"
is asking for a v2 version offoo
.
semver is a foundation of the modules system. In order to provide the best experience for consumers, module authors are encouraged to apply semver VCS tags (e.g., v0.1.0
or v1.2.3-rc.1
), but semver VCS tags are not strictly required:
-
Modules are required to follow the semver specification in order for the
go
command to behave as documented. This includes following the semver specification regarding how and when breaking changes are allowed. -
Modules that do not have semver VCS tags are recorded by consumers using a semver version in the form of a pseudo-version. Typically this will be a v0 major version, unless the module author constructed a v2+ module following the "Major Subdirectory" approach.
-
Therefore, modules that do not apply semver VCS tags and have not created a "Major Subdirectory" are effectively declaring themselves to be in the semver v0 major version series, and a module-based consumer will treat them as having a semver v0 major version.
A module can depend on a different major version of itself: by-and-large, this is comparable to depending on a different module. This can be useful for different reasons, including to allow a major version of a module to be implemented as a shim around a different major version.
In addition, a module can depend on a different major version of itself in a cycle, just as two completely different modules can depend on each other in a cycle.
However, if you are not expecting a module to depend on a different version of itself, it can be a sign of a mistake. For example, .go code intending to import a package from a v3 module might be missing the required /v3
in the import statement. That mistake can manifest as a v3 module depending on the v1 version of itself.
If you are surprised to see a module to depend on a different version of itself, it can be worthwhile to review the "Semantic Import Versioning" section above along with the FAQ "What can I check if I am not seeing the expected version of a dependency?".
It continues to be a constraint that two packages may not depend on each other in a cycle.
A multi-module repository is a repository that contains multiple modules, each with its own go.mod file. Each module starts at the directory containing its go.mod file, and contains all packages from that directory and its subdirectories recursively, excluding any subtree that contains another go.mod file.
Each module has its own version information. Version tags for modules below the root of the repository must include the relative directory as a prefix. For example, consider the following repository:
my-repo
|____foo
| |____rop
| | |____go.mod
The tag for version 1.2.3 of module "my-repo/foo/rop" is "foo/rop/v1.2.3".
Typically, the path for one module in the repository will be a prefix of the others. For example, consider this repository:
my-repo
|____go.mod
|____bar
|____foo
| |____rop
| |____yut
|____mig
| |____go.mod
| |____vub
Fig. A top-level module's path is a prefix of another module's path.
This repository contains two modules. However, the module "my-repo" is a prefix of the path of the module "my-repo/mig".
Adding modules, removing modules, and versioning modules in such a configuration require considerable care and deliberation, so it is almost always easier and simpler to manage a single-module repository rather than multiple modules in an existing repository.
Russ Cox commented in #26664:
For all but power users, you probably want to adopt the usual convention that one repo = one module. It's important for long-term evolution of code storage options that a repo can contain multiple modules, but it's almost certainly not something you want to do by default.
Two examples of how multi-modules can be more work:
-
go test ./...
from the repository root will no longer test everything in the repository - you might need to routinely manage the relationship between the modules via
replace
directives.
However, there is additional nuance beyond those two examples. Please read the FAQs in this sub-section carefully if you are considering having multiple modules in a single repository.
Two example scenarios where it can make sense to have more than one go.mod
in a repository:
-
if you have usage examples where the examples themselves have a complex set of dependencies (e.g., perhaps you have a small package but include an example of using your package with kubernetes). In that case, it can make sense for your repository to have an
examples
or_examples
directory with its owngo.mod
, such as shown here. -
if you have a repository with a complex set of dependencies, but you have a client API with a smaller set of dependencies. In some cases, it might make sense to have an
api
orclientapi
or similar directory with its owngo.mod
, or to separate out thatclientapi
into its own repository.
However, for both of those cases, if you are considering creating a multi-module repository for performance or download size for a large set of indirect dependencies, you are strongly encouraged to first try with a GOPROXY, which will be enabled by default in Go 1.13. Using a GOPROXY mostly equals any performance benefits or dependency download size benefits that might otherwise come from creating a multi-module repository.
Yes. However, there are two classes of this problem:
The first class: the package to which the module is being added to is not in version control yet (a new package). This case is straightforward: add the package and the go.mod in the same commit, tag the commit, and push.
The second class: the path at which the module is being added is in version control and contains one or more existing packages. This case requires a considerable amount of care. To illustrate, consider again the following repository (now in a www.greatytc.com location to simulate the real-world better):
www.greatytc.com/my-repo
|____go.mod
|____bar
|____foo
| |____rop
| |____yut
|____mig
| |____vub
Consider adding module "www.greatytc.com/my-repo/mig". If one were to follow the same approach as above, the package /my-repo/mig could be provided by two different modules: the old version of "www.greatytc.com/my-repo", and the new, standalone module "www.greatytc.com/my-repo/mig. If both modules are active, importing "www.greatytc.com/my-repo/mig" would cause an “ambiguous import” error at compile time.
The way to get around this is to make the newly-added module depend on the module it was "carved out" from, at a version after which it was carved out.
Let's step through this with the above repository, assuming that "www.greatytc.com/my-repo" is currently at v1.2.3:
-
Add www.greatytc.com/my-repo/mig/go.mod:
cd path-to/www.greatytc.com/my-repo/mig go mod init www.greatytc.com/my-repo/mig # Note: if "my-repo/mig" does not actually depend on "my-repo", add a blank # import. # Note: version must be at or after the carve-out. go mod edit -require www.greatytc.com/myrepo@v1.3
-
git commit
-
git tag v1.3.0
-
git tag mig/v1.0.0
-
Next, let's test these. We can't
go build
orgo test
naively, since the go commands would try to fetch each dependent module from the module cache. So, we need to use replace rules to causego
commands to use the local copies:cd path-to/www.greatytc.com/my-repo/mig go mod edit -replace www.greatytc.com/my-repo@v1.3.0=../ go test ./... go mod edit -dropreplace www.greatytc.com/my-repo@v1.3.0
-
git push origin master v1.2.4 mig/v1.0.0
push the commit and both tags
Note that in the future golang.org/issue/28835 should make the testing step a more straightforward experience.
Note also that code was removed from module "www.greatytc.com/my-repo" between minor versions. It may seem strange to not consider this a major change, but in this instance the transitive dependencies continue to provide compatible implementations of the removed packages at their original import paths.
Yes, with the same two cases and similar steps as above.
Yes. Packages in one module are allowed to import internal packages from another module as long as they share the same path prefix up to the internal/ path component. For example, consider the following repository:
my-repo
|____go.mod
|____internal
|____foo
| |____go.mod
Here, package foo can import /my-repo/internal as long as module "my-repo/foo" depends on module "my-repo". Similarly, in the following repository:
my-repo
|____internal
| |____go.mod
|____foo
| |____go.mod
Here, package foo can import my-repo/internal as long as module "my-repo/foo" depends on module "my-repo/internal". The semantics are the same in both: since my-repo is a shared path prefix between my-repo/internal and my-repo/foo, package foo is allowed to import package internal.
Can an additional go.mod exclude unnecessary content? Do modules have the equivalent of a .gitignore file?
One additional use case for having multiple go.mod
files in a single repository is if the repository has files that should be pruned from a module. For example, a repository might have very large files that are not needed for the Go module, or a multi-language repository might have many non-Go files.
An empty go.mod
in a directory will cause that directory and all of its subdirectories to be excluded from the top-level Go module.
If the excluded directory does not contain any .go
files, no additional steps are needed beyond placing the empty go.mod
file. If the excluded directory does contain .go
files, please first carefully review the other FAQs in this multi-module repository section.
Please see the question "Won't minimal version selection keep developers from getting important updates?" in the earlier FAQ from the official proposal discussion.
- Double-check that modules are enabled by running
go env
to confirm it does not show an empty value for the read-onlyGOMOD
variable.- Note: you never set
GOMOD
as a variable because it is effectively read-only debug output thatgo env
outputs. - If you are setting
GO111MODULE=on
to enable modules, double-check that it is not accidentally the pluralGO111MODULES=on
. (People sometimes naturally include theS
because the feature is often called "modules").
- Note: you never set
- If vendoring is expected to be used, double-check check that the
-mod=vendor
flag is being passed togo build
or similar, or thatGOFLAGS=-mod=vendor
is set.- Modules by default ignore the
vendor
directory unless you ask thego
tool to usevendor
.
- Modules by default ignore the
- It is frequently helpful to check
go list -m all
to see the list of actual versions selected for your build-
go list -m all
usually gives you more detail compared to if you were to instead just look ago.mod
file.
-
- If running
go get foo
fails in some way, or ifgo build
is failing on a particular packagefoo
, it often can be helpful to check the output fromgo get -v foo
orgo get -v -x foo
:- In general,
go get
will often provide more a detailed error message thango build
. - The
-v
flag togo get
asks to print more verbose details, though be mindful that certain "errors" such as 404 errors might be expected based on how a remote repository was configured. - If the nature of the problem is still not clear, you can also try the more verbose
go get -v -x foo
, which also shows the git or other VCS commands being issued. (If warranted, you can often execute the same git commands outside of the context of thego
tool for troubleshooting purposes).
- In general,
- You can check to see if you are using a particularly old git version
- Older versions of git were a common source of problems for the
vgo
prototype and Go 1.11 beta, but much less frequently in the GA 1.11.
- Older versions of git were a common source of problems for the
- The module cache in Go 1.11 can sometimes cause various errors, primarily if there were previously network issues or multiple
go
commands executing in parallel (see #26794, which is addressed for Go 1.12). As a troubleshooting step, you can copy $GOPATH/pkg/mod to a backup directory (in case further investigation is warranted later), rungo clean -modcache
, and then see whether the original problem persists. - If you are using Docker, it can be helpful to check if you can reproduce the behavior outside of Docker (and if the behavior only occurs in Docker, the list of bullets above can be used as a starting point to compare results between inside Docker vs. outside).
The error you are currently examining might be a secondary issue caused by not having the expected version of a particular module or package in your build. Therefore, if the cause of a particular error is not obvious, it can be helpful to spot check your versions as described in the next FAQ.
-
A good first step is to run
go mod tidy
. There is some chance this might resolve the issue, but it will also help put yourgo.mod
file into a consistent state with respect to your.go
source code, which will help make any subsequent investigation easier. (Ifgo mod tidy
itself changes the versions of a dependency in a way you don't expect, first read this FAQ on 'go mod tidy'. If that does not explain it, you can try resetting yourgo.mod
and then rungo list -mod=readonly all
, which might give a more specific message about whatever was requiring a change to its version). -
The second step usually should be to check
go list -m all
to see the list of actual versions selected for your build.go list -m all
shows you the final selected versions, including for indirect dependencies and after resolving versions for any shared dependencies. It also shows the outcome of anyreplace
andexclude
directives. -
A good next step can be to examine the output of
go mod graph
orgo mod graph | grep <module-of-interest>
.go mod graph
prints the module requirement graph (including taking into account replacements). Each line in the output has two fields: the first column is a consuming module, and the second column is one of that module's requirements (including the version required by that consuming module). This can be a quick way to see which modules are requiring a particular dependency, including when your build has a dependency that has different required versions from different consumers in your build (and if that is the case, it is important to be familiar with the behavior described in the "Version Selection" section above).
go mod why -m <module>
can also be useful here, although it is typically more useful for seeing why a dependency is included at all (rather than why a dependency ends up with a particular version).
go list
provides many more variations of queries that can be useful to interrogate your modules if needed. One example is the following, which will show the exact versions used in your build excluding test-only dependencies:
go list -deps -f '{{with .Module}}{{.Path}} {{.Version}}{{end}}' ./... | sort -u
A more detailed set of commands and examples for interrogating your modules can be seen in a runnable "Go Modules by Example" walkthough.
One cause of unexpected versions can be due to someone having created an invalid or unexpected go.mod
file that was not intended, or a related mistake (for example: a v2.0.1
version of module might have incorrectly declared itself to be module foo
in its go.mod
without the required /v2
; an import statement in .go
code intended to import a v3 module might be be missing the required /v3
; a require
statement in a go.mod
for a v4 module might be be missing the required /v4
). Therefore, if the cause of a particular issue you are seeing is not obvious, it can be worthwhile to first re-read the material in the "go.mod" and "Semantic Import Versioning" sections above (given these include important rules that modules must follow) and then take a few minutes to spot check the most relevant go.mod
files and import statements.
This is a general error message that can occur for several different underlying causes.
In some cases, this error is simply due to a mistyped path, so the first step likely should be to double-check for incorrect paths based on the details listed in the error message.
If you have not already done so, a good next step is often to try go get -v foo
or go get -v -x foo
:
- In general,
go get
will often provide more a detailed error message thango build
. - See the first troubleshooting FAQ in this section above for more details.
Some other possible causes:
-
You might see the error
cannot find module providing package foo
if you have issuedgo build
orgo build .
but do not have any.go
source files in the current directory. If this is what you are encountering, the solution might be an alternative invocation such asgo build ./...
(where the./...
expands out to match all the packages within the current module). See #27122. -
The module cache in Go 1.11 can cause this error, including in the face of network issues or multiple
go
commands executing in parallel. This is resolved in Go 1.12. See the first troubleshooting FAQ in this section above for more details and possible corrective steps.
go mod init
without any arguments will attempt to guess the proper module path based on different hints such as VCS meta data. However, it is not expected that go mod init
will always be able to guess the proper module path.
If go mod init
gives you this error, those heuristics were not able to guess, and you must supply the module path yourself (such as go mod init www.greatytc.com/you/hello
).
I have a problem with a complex dependency that has not opted in to modules. Can I use information from its current dependency manager?
Yes. This requires some manual steps, but can be helpful in some more complex cases.
When you run go mod init
when initializing your own module, it will automatically convert from a prior dependency manager by translating configuration files like Gopkg.lock
, glide.lock
, or vendor.json
into a go.mod
file that contains corresponding require
directives. The information in a pre-existing Gopkg.lock
file for example usually describes version information for all of your direct and indirect dependencies.
However, if instead you are adding a new dependency that has not yet opted in to modules itself, there is not a similar automatic conversion process from any prior dependency manager that your new dependency might have been using. If that new dependency itself has non-module dependencies that have had breaking changes, then in some cases that can cause incompatibility problems. In other words, a prior dependency manager of your new dependency is not automatically used, and that can cause problems with your indirect dependencies in some cases.
One approach is to run go mod init
on your problematic non-module direct dependency to convert from its current dependency manager, and then use the require
directives from the resulting temporary go.mod
to populate or update the go.mod
in your module.
For example, if www.greatytc.com/some/nonmodule
is a problematic direct dependency of your module that is currently using another dependency manager, you can do something similar to:
$ git clone -b v1.2.3 //www.greatytc.com/some/nonmodule /tmp/scratchpad/nonmodule
$ cd /tmp/scratchpad/nonmodule
$ go mod init
$ cat go.mod
The resulting require
information from the temporary go.mod
can be manually moved into the actual go.mod
for your module, or you can consider using //www.greatytc.com/rogpeppe/gomodmerge, which is a community tool targeting this use case. In addition, you will want to add a require www.greatytc.com/some/nonmodule v1.2.3
to your actual go.mod
to match the version that you manually cloned.
A concrete example of following this technique for docker is in this #28489 comment, which illustrates getting a consistent set of versions
of docker dependencies to avoid case sensitive issues between www.greatytc.com/sirupsen/logrus
vs. www.greatytc.com/Sirupsen/logrus
.
How can I resolve "parsing go.mod: unexpected module path" and "error loading module requirements" errors caused by a mismatch between import paths vs. declared module identity?
In general, a module declares its identity in its go.mod
via the module
directive, such as module example.com/m
. This is the "module path" for that module, and the go
tool enforces consistency between that declared module path and the import paths used by any consumer. If a module's go.mod
file reads module example.com/m
, then a consumer must import packages from that module using import paths that start with that module path (e.g., import "example.com/m"
or import "example.com/m/sub/pkg"
).
The go
command reports a parsing go.mod: unexpected module path
fatal error if there is a mismatch between an import path used by a consumer vs. the corresponding declared module path. In addition, in some cases the go
command will then report a more generic error loading module requirements
error afterwards.
The most common cause of this error is if there was a name change (e.g., www.greatytc.com/Sirupsen/logrus
to www.greatytc.com/sirupsen/logrus
), or if a module was sometimes used via two different names prior to modules due to a vanity import path (e.g., www.greatytc.com/golang/sync
vs. the recommended golang.org/x/sync
).
This can then cause problems if you have a dependency that is still being imported via an older name (e.g., www.greatytc.com/Sirupsen/logrus
) or a non-canonical name (e.g., www.greatytc.com/golang/sync
) but that dependency has subsequently adopted modules and now declares its canonical name in its go.mod
. The error here can then trigger during an upgrade when the upgraded version of the module is found declaring a canonical module path that no longer matches the older import path.
- You are indirectly depending on
www.greatytc.com/Quasilyte/go-consistent
. - The project adopts modules, and then later changes its name to
www.greatytc.com/quasilyte/go-consistent
(changingQ
to lowercaseq
), which is a breaking change. GitHub forwards from the old name to the new name. - You run
go get -u
, which attempts to upgrade all of your direct and indirect dependencies. -
www.greatytc.com/Quasilyte/go-consistent
is attempted to be upgraded, but the latestgo.mod
found now readsmodule www.greatytc.com/quasilyte/go-consistent
. - The overall upgrade operation fails to complete, with error:
go: www.greatytc.com/Quasilyte/go-consistent@v0.0.0-20190521200055-c6f3937de18c: parsing go.mod: unexpected module path "www.greatytc.com/quasilyte/go-consistent" go get: error loading module requirements
The most common form of the error is:
go: example.com/some/OLD/name@vX.Y.Z: parsing go.mod: unexpected module path "example.com/some/NEW/name"
If you visit the repository for example.com/some/NEW/name
(from the right-side of the error), you can check the go.mod
file for the latest release or master
to see if it declares itself on the first line of the go.mod
as module example.com/some/NEW/name
. If so, that is a hint that you are seeing an "old module name" vs. "new module name" problem.
This remainder of this section focuses on resolving the "old name" vs. "new name" form of this the error by following these steps in sequence:
-
Check your own code to see if you are importing using
example.com/some/OLD/name
. If so, update your code to import usingexample.com/some/NEW/name
. -
If you received this error during an upgrade, you should try upgrading using the tip version of Go, which has more targeted upgrade logic (#26902) that can often sidestep this problem and also often has a better error message for this situation. Note that the
go get
arguments in tip / 1.13 are different than in 1.12. Example of obtaining tip and using it to upgrade your dependencies:
go get golang.org/dl/gotip && gotip download
gotip get -u all
gotip mod tidy
Because the problematic old import is often in an indirect dependency, upgrading with tip and then running go mod tidy
can frequently upgrade you past the problematic version and then also remove the problematic version from your go.mod
as no longer needed, which then puts you into a functioning state when you return to using Go 1.12 or 1.11 for day-to-day use. For example, see that approach work here to upgrade past www.greatytc.com/golang/lint
vs. golang.org/x/lint
problems.
-
If you received this error while doing
go get -u foo
orgo get -u foo@latest
, try removing the-u
. This will give you the set of dependencies used byfoo@latest
without upgrading the dependencies offoo
past the versions that the author offoo
likely verified as working when releasingfoo
. This can be important especially during this transitional time when some of the direct and indirect dependencies offoo
might not yet have adopted semver or modules. (A common mistake is thinkinggo get -u foo
solely gets the latest version offoo
. In actuality, the-u
ingo get -u foo
orgo get -u foo@latest
means to also get the latest versions for all of the direct and indirect dependencies offoo
; that might be what you want, but it might not be especially if it is otherwise failing due to deep indirect dependencies). -
If the steps above have not resolved the error, the next approach is slightly more complicated, but most often should work to resolve an "old name" vs. "new name" form of this error. This uses just information solely from the error message itself, plus some brief looking at some VCS history.
4.1. Go to the
example.com/some/NEW/name
repository4.2. Determine when the
go.mod
file was introduced there (e.g., by looking at the blame or history view for thego.mod
).4.3. Pick the release or commit from just before the
go.mod
file was introduced there.4.4. In your
go.mod
file, add areplace
statement using the old name on both sides of thereplace
statement:replace example.com/some/OLD/name => example.com/some/OLD/name <version-just-before-go.mod>
Using our prior example wherewww.greatytc.com/Quasilyte/go-consistent
is the old name andwww.greatytc.com/quasilyte/go-consistent
is the new name, we can see that thego.mod
was first introduced there in commit 00c5b0cf371a. That repository is not using semver tags, so we will take the immediately prior commit 00dd7fb039e and add it to the replace using the old uppercase Quasilyte name on both sides of thereplace
:
replace www.greatytc.com/Quasilyte/go-consistent => www.greatytc.com/Quasilyte/go-consistent 00dd7fb039e
This replace
statement then enables us to upgrade past the problematic "old name" vs. "new name" mismatch by effectively preventing the old name from being upgraded to the new name in the presence of a go.mod
. Usually, an upgrade via go get -u
or similar can now avoid the error. If the upgrade completes, you can check to see if anyone is still importing the old name (e.g., go mod graph | grep www.greatytc.com/Quasilyte/go-consistent
) and if not, the replace
can then be removed. (The reason this often works is because the upgrade itself can otherwise fail if an old problematic import path is used even though it might not be used in the final result if the upgrade had completed, which is tracked in #30831).
- If the above steps have not resolved the problem, it might be because the problematic old import path is still in use by the latest version of one or more of your dependencies. In this case, it is important to identify who is still using the problematic old import path, and find or open an issue asking that the problematic importer change to using the now canonical import path. Using
gotip
in step 2. above might identify the problematic importer, but it does not do so in all cases, especially for upgrades (#30661). If it is unclear who is importing using the problematic old import path, you can usually find out by creating a clean module cache, performing the operation or operations that trigger the error, and then grepping for the old problematic import path within the module cache. For example:
export GOPATH=$(mktemp -d)
go get -u foo # peform operation that generates the error of interest
cd $GOPATH/pkg/mod
grep -R --include="*.go" www.greatytc.com/Quasilyte/go-consistent
- If these steps are not sufficient to resolve the issue, or if you are a maintainer of a project that seems unable to remove references to an older problematic import path due to circular references, please see a much more detailed write-up of the problem on a separate wiki page.
Finally, the above steps focus on how to resolve an underlying "old name" vs. "new name" problem. However, the same error message can also appear if a go.mod
was placed in the wrong location or simply has the wrong module path. If that is the case, the importing that module should always fail. If you are importing a new module that you just created and has never been successfully imported before, you should check that the go.mod
file is located correctly and that it has the proper module path that corresponds to that location. (The most common approach is a single go.mod
per repository, with the single go.mod
file placed in the repository root, and using the repository name as the module path declared in the module
directive). See the "go.mod" section for more details.
In short:
Because the pre-built packages are non-module builds and can’t be reused. Sorry. Disable cgo for now or install gcc.
This is only an issue when opting in to modules (e.g., via GO111MODULE=on
). See #26988 for additional discussion.
No. See #26645, which includes:
In modules, there finally is a name for the subdirectory. If the parent directory says "module m" then the subdirectory is imported as "m/subdir", no longer "./subdir".
Directories without .go
files are not copied inside the vendor
directory by go mod vendor
. This is by design.
In short, setting aside any particular vendoring behavior – the overall model for go builds is that the files needed to build a package should be in the directory with the .go
files.
Using the example of cgo – modifying C source code in other directories will not trigger a rebuild, and instead your build will use stale cache entries. The cgo documentation now includes:
Note that changes to files in other directories do not cause the package to be recompiled, so all non-Go source code for the package should be stored in the package directory, not in subdirectories.
A community tool //www.greatytc.com/goware/modvendor allows you to easily copy a complete set of .c, .h, .s, .proto or other files from a module into the vendor
director. Although this can be helpful, some care must be taken to make sure your go build is being handled properly in general (regardless of vendoring) if you have files needed to build a package that are outside of the directory with the .go
files.
See additional discussion in #26366.
An alternative approach to traditional vendoring is to check in the module cache. It can end up with similar benefits as traditional vendoring and in some ways ends up with a higher fidelity copy. This approach is explained as a "Go Modules by Example" walkthrough.