Golang 1.13 module mirror, index, and Checksum database are now production-ready

Last week, the Golang team announced that the Go module mirror, index, and checksum database are now production-ready thus adding reliability and security to the Go ecosystem. For Go 1.13 module users, the go command will use the module mirror and checksum database by default.

New production-ready modules for Go 1.13 module

Module Mirror

A module mirror is a special kind of module proxy that caches metadata and source code in its own storage system. This allows the mirror to continue to serve source code that is no longer available from the original locations thus speeding up downloads and protect users from the disappearing dependencies.

According to the team, module mirror is served at proxy.golang.org, which the go command will use by default for module users as of Go 1.13. For users still running an earlier version of the go command, they can use this service by setting GOPROXY=https://proxy.golang.org in their local environment.

Read Also: The Go team shares new proposals planned to be implemented in Go 1.13 and 1.14

Module Index

The module index is served by index.golang.org. It is a public feed of new module versions that become available through proxy.golang.org. Module index is useful for tool developers who want to keep their own cache of what’s available in proxy.golang.org, or to keep up-to-date on some of the newest modules go developers use.

Read Also: Implementing Garbage collection algorithms in Golang [Tutorial]

Checksum Database

Modules introduced the go.sum file, a list of SHA-256 hashes of the source code and go.mod files of each dependency when it was first downloaded. The go command can use these hashes to detect misbehavior by an origin server or proxy that gives a different code for the same version. However, the go.sum file has a limitation, it works entirely by trust based on user’s first use. When a user adds a version of a never seen before dependency, the go command fetches the code and adds lines to the go.sum file quickly. The problem is that those go.sum lines aren’t being checked against anyone else’s and thus they might be different from the go.sum lines that the go command just generated for someone else.

The checksum database ensures that the go command always adds the same lines to everyone's go.sum file. Whenever the go command receives new source code, it can verify the hash of that code against this global database to make sure the hashes match, ensuring that everyone is using the same code for a given version.

The checksum database is served by sum.golang.org and is built on a Transparent Log (or “Merkle tree”) of hashes backed by Trillian, a transparent, highly scalable and cryptographically verifiable data store. The main advantage of a Merkle tree is that it is tamper-proof and has properties that don’t allow for misbehavior to go undetected, making it more trustworthy. The Merkle tree checks inclusion proofs (if a specific record exists in the log) and “consistency” proofs (that the tree hasn’t been tampered with) before adding new go.sum lines to a user’s module’s go.sum file.

This checksum database allows the go command to safely use an otherwise untrusted proxy. Because there is an auditable security layer sitting on top of it, a proxy or origin server can’t intentionally, arbitrarily, or accidentally start giving you the wrong code without getting caught.

“Even the author of a module can’t move their tags around or otherwise change the bits associated with a specific version from one day to the next without the change being detected,” the blog mentions.

Developers are excited about the launch of the module mirror and checksum database and look forward to checking it out.

https://twitter.com/hasdid/status/1167795923944124416

https://twitter.com/jedisct1/status/1167183027283353601

To know more about this news in detail, read the official blog post.