docker-buildx: init at 0.5.1

[bobrik]

Motivation for this change

Installing docker-buildx enables buildx subcommand on the client:

• https://github.com/docker/buildx

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS (aarch64)
  • nixos/nix docker image (aarch64)
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using (n/a)
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[bobrik]

This change requires the following in ~/.docker/config.json to make it work for me:

{
    "experimental": "enabled",
    "cliPluginsExtraDirs": [
        "/Users/ivan/.nix-profile/libexec/docker/cli-plugins"
    ]
}

Here /User/ivan is my $HOME on macOS.

Ideally I'd like it to work outside of the box, but even the following patch doesn't do the trick:

• https://github.com/bobrik/docker-ce/commit/a03bb6bdfdf115951fbff023cc615270a661af3c.patch

I'd love some pointers on how things like that are supposed to work with Nix.

[r-burns]

Welcome to nixpkgs! :)

Since you are adding yourself as a maintainer, this is typically done in two separate commits:

1. add yourself to the maintainer-list
2. add the package, with yourself as a maintainer

I'm not sure how the plugin automation would work on darwin - there might be issues with purity: LnL7/nix-darwin#112
If you'd like to set it up on NixOS though, that should be possible by modifying nixos/modules/virtualisation/docker.nix.

[bobrik]

I've split my changes into two commits as requested.

Not sure if LnL7/nix-darwin#112 is relevant here, since I'm adding buildx support on the client. Ultimately I want Docker client to be able to find $HOME/.nix-profile/libexec/docker/cli-plugins/docker-buildx when I type docker buildx the same way man curl is able to find $HOME/.nix-profile/share/man/man1/curl.1.gz. Any pointers describing how this sort of thing is implemented with Nix would be appreciated.

[r-burns]

Hm you are right I thought there was some mention of config.json issues there. I'm surprised the buildx command doesn't just work - e.g. for git-lfs it's just a separate binary that shows up as a subcommand along with the other git-* binaries. Does buildx need extra setup?

[bobrik]

In Git's case it's a matter of running git-something from $PATH when git something is invoked:

$ echo -e "#!""$(which bash)\necho howdy" | sudo tee /usr/local/bin/git-ivan && sudo chmod +x /usr/local/bin/git-ivan
#!/Users/ivan/.nix-profile/bin/bash
echo howdy

$ git ivan
howdy

With Docker it tries to look up the binary in the list of hardcoded (+configured) paths:

• https://github.com/docker/docker-ce/blob/master/components/cli/cli-plugins/manager/manager_unix.go#L5-L8

And $HOME/.nix-profile/libexec/docker/cli-plugins is not in this list.

[r-burns]

Well I think checking only the nix-profile might run into issues with nix-shell or system installations - can docker consume a list of plugin directories from e.g. an environment variable? If not, how about something like docker.withPlugins to construct an env with specified plugins on the fly, similar to how python.withPackages works?

[bobrik]

I managed to add buildxSupport argument to docker package in my last commit and it seems to work as intended. This is similar in spirit to what git does with sendEmailSupport flag.

With the following in my ~/.nixpkgs/config.nix:

{
  packageOverrides = pkgs: {
    docker = pkgs.docker.override { buildxSupport = true; };
  };
}

Building docker from my local copy of nixpkgs:

$ nix-env --file . --install docker

Produces the docker client binary that know to look for docker-buildx plugin in the right place:

• /nix/store/33halip2fz0m844d3v1sq6mzm5lrjfic-docker-buildx-0.5.1/libexec/docker/cli-plugins/docker-buildx

$ docker buildx

Usage:  docker buildx [OPTIONS] COMMAND

Build with BuildKit

Options:
      --builder string   Override the configured builder instance

Management Commands:
  imagetools  Commands to work on images in registry

Commands:
  bake        Build from a file
  build       Start a build
  create      Create a new builder instance
  du          Disk usage
  inspect     Inspect current builder instance
  ls          List builder instances
  prune       Remove build cache
  rm          Remove a builder instance
  stop        Stop builder instance
  use         Set the current builder instance
  version     Show buildx version information

Run 'docker buildx COMMAND --help' for more information on a command.

With the only caveat that user has to enable experimental features in ~/.docker/config.json:

{
    "experimental": "enabled"
}

[bobrik]

cc docker package maintainers: @offlinehacker, @tailhook, @vdemeester, @periklis

[r-burns]

Nice approach 👍

One caveat of this over something like withPlugins is that you'll have to rebuild all of docker when you ask for buildx support, rather than just the wrapper. But if there's no way to specify the plugin dirs at runtime, I guess this is unfortunately unavoidable.

An alternative interface for the override could be to accept a list of plugins to add to the search path, but if buildx is the only plugin we need to care about right now, then a simple boolean toggle is probably more straightforward.

[bobrik]

@r-burns, I amended my commits with your changes 👍

[r-burns]

Thanks, nice work 👍

I'd like a second opinion from one of the docker package maintainers, but this all looks good to me

[vdemeester]

LGTM 🐯

[bobrik]

Is there anything else I should do to get this merged?

[r-burns]

/marvin opt-in
/status needs_merger

[SuperSandro2000]

as far as I can tell the diff LGTM

[SuperSandro2000]

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.

Result of nixpkgs-review pr 107302 run on x86_64-linux 1

1 package built:

• docker-buildx

[SuperSandro2000]

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.

Result of nixpkgs-review pr 107302 run on x86_64-darwin 1

1 package built:

• docker-buildx

[bobrik]

Any reason not to merge?

[SuperSandro2000]

Any reason not to merge?

The merge conflict and the thins I noted above.

[bobrik]

Rebased on top of the latest master branch.

[bobrik]

Resolved conflicts by rebasing once again.

[SuperSandro2000]

This is a semi-automatic executed nixpkgs-review which is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.

Result of nixpkgs-review pr 107302 run on x86_64-darwin 1

1 package built:

• docker-buildx

[bobrik]

Small correction was needed after a rebase: #110543.