Skip to content

Commit 59b5ed6

Browse files
tonistiigitonistiigi
authored
Merge pull request #165 from crazy-max/append
append nodes to builder support
2 parents 95cb08c + bd61d52 commit 59b5ed6

15 files changed

+365
-36
lines changed

‎.github/workflows/ci.yml

+48
Original file line numberDiff line numberDiff line change
@@ -305,9 +305,13 @@ jobs:
305305
platforms: ${{ matrix.qemu-platforms }}
306306
-
307307
name: Set up Docker Buildx
308+
id: buildx
308309
uses: ./
309310
with:
310311
version: ${{ matrix.buildx-version }}
312+
-
313+
name: List builder platforms
314+
run: echo ${{ steps.buildx.outputs.platforms }}
311315

312316
build-ref:
313317
runs-on: ubuntu-latest
@@ -416,3 +420,47 @@ jobs:
416420
echo "::error::Should have failed"
417421
exit 1
418422
fi
423+
424+
append:
425+
runs-on: ubuntu-latest
426+
steps:
427+
-
428+
name: Checkout
429+
uses: actions/checkout@v3
430+
-
431+
name: Create dummy contexts
432+
run: |
433+
docker context create ctxbuilder2
434+
docker context create ctxbuilder3
435+
-
436+
name: Set up Docker Buildx
437+
id: buildx
438+
uses: ./
439+
with:
440+
append: |
441+
- name: builder2
442+
endpoint: ctxbuilder2
443+
platforms: linux/amd64
444+
driver-opts:
445+
- image=moby/buildkit:master
446+
- network=host
447+
- endpoint: ctxbuilder3
448+
platforms: linux/arm64
449+
-
450+
name: List builder platforms
451+
run: echo ${{ steps.buildx.outputs.platforms }}
452+
453+
platforms:
454+
runs-on: ubuntu-latest
455+
steps:
456+
-
457+
name: Checkout
458+
uses: actions/checkout@v3
459+
-
460+
name: Set up QEMU
461+
uses: docker/setup-qemu-action@v2
462+
-
463+
name: Set up Docker Buildx
464+
uses: ./
465+
with:
466+
platforms: linux/amd64

‎README.md

+25-19
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ ___
2121
* [Usage](#usage)
2222
* [Advanced usage](#advanced-usage)
2323
* [Authentication support](docs/advanced/auth.md)
24+
* [Append additional nodes to the builder](docs/advanced/append-nodes.md)
2425
* [Install by default](docs/advanced/install-default.md)
2526
* [BuildKit daemon configuration](docs/advanced/buildkit-config.md)
2627
* [Standalone mode](docs/advanced/standalone.md)
@@ -61,6 +62,7 @@ jobs:
6162
## Advanced usage
6263
6364
* [Authentication support](docs/advanced/auth.md)
65+
* [Append additional nodes to the builder](docs/advanced/append-nodes.md)
6466
* [Install by default](docs/advanced/install-default.md)
6567
* [BuildKit daemon configuration](docs/advanced/buildkit-config.md)
6668
* [Standalone mode](docs/advanced/standalone.md)
@@ -69,32 +71,36 @@ jobs:
6971
7072
### inputs
7173
72-
Following inputs can be used as `step.with` keys
74+
Following inputs can be used as `step.with` keys:
7375

74-
| Name | Type | Description |
75-
|-------------------|--------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
76-
| `version` | String | [Buildx](https://github.com/docker/buildx) version. (eg. `v0.3.0`, `latest`, `https://github.com/docker/buildx.git#master`) |
77-
| `driver` | String | Sets the [builder driver](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver) to be used (default `docker-container`) |
78-
| `driver-opts` | CSV | List of additional [driver-specific options](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver-opt) (eg. `image=moby/buildkit:master`) |
79-
| `buildkitd-flags` | String | [Flags for buildkitd](https://docs.docker.com/engine/reference/commandline/buildx_create/#buildkitd-flags) daemon (since [buildx v0.3.0](https://github.com/docker/buildx/releases/tag/v0.3.0)) |
80-
| `install` | Bool | Sets up `docker build` command as an alias to `docker buildx` (default `false`) |
81-
| `use` | Bool | Switch to this builder instance (default `true`) |
82-
| `endpoint` | String | [Optional address for docker socket](https://docs.docker.com/engine/reference/commandline/buildx_create/#description) or context from `docker context ls` |
83-
| `config`¹ | String | [BuildKit config file](https://docs.docker.com/engine/reference/commandline/buildx_create/#config) |
84-
| `config-inline`¹ | String | Same as `config` but inline |
85-
86-
> * ¹ `config` and `config-inline` are mutually exclusive
87-
88-
> `CSV` type must be a newline-delimited string
89-
> ```yaml
90-
> driver-opts: image=moby/buildkit:master
91-
> ```
76+
> `List` type is a newline-delimited string
9277
> ```yaml
9378
> driver-opts: |
9479
> image=moby/buildkit:master
9580
> network=host
9681
> ```
9782

83+
> `CSV` type must be a newline-delimited string
84+
> ```yaml
85+
> platforms: linux/amd64,linux/arm64
86+
> ```
87+
88+
| Name | Type | Description |
89+
|-------------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
90+
| `version` | String | [Buildx](https://github.com/docker/buildx) version. (eg. `v0.3.0`, `latest`, `https://github.com/docker/buildx.git#master`) |
91+
| `driver` | String | Sets the [builder driver](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver) to be used (default `docker-container`) |
92+
| `driver-opts` | List | List of additional [driver-specific options](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver-opt) (eg. `image=moby/buildkit:master`) |
93+
| `buildkitd-flags` | String | [Flags for buildkitd](https://docs.docker.com/engine/reference/commandline/buildx_create/#buildkitd-flags) daemon (since [buildx v0.3.0](https://github.com/docker/buildx/releases/tag/v0.3.0)) |
94+
| `install` | Bool | Sets up `docker build` command as an alias to `docker buildx` (default `false`) |
95+
| `use` | Bool | Switch to this builder instance (default `true`) |
96+
| `endpoint` | String | [Optional address for docker socket](https://docs.docker.com/engine/reference/commandline/buildx_create/#description) or context from `docker context ls` |
97+
| `platforms` | List/CSV | Fixed [platforms](https://docs.docker.com/engine/reference/commandline/buildx_create/#platform) for current node. If not empty, values take priority over the detected ones. |
98+
| `config`¹ | String | [BuildKit config file](https://docs.docker.com/engine/reference/commandline/buildx_create/#config) |
99+
| `config-inline`¹ | String | Same as `config` but inline |
100+
| `append` | YAML | [Append additional nodes](docs/advanced/append-nodes.md) to the builder |
101+
102+
> * ¹ `config` and `config-inline` are mutually exclusive
103+
98104
### outputs
99105

100106
Following outputs are available

‎__tests__/context.test.ts

+70
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import * as os from 'os';
44
import * as path from 'path';
55
import * as uuid from 'uuid';
66
import * as context from '../src/context';
7+
import * as nodes from '../src/nodes';
78

89
const tmpdir = fs.mkdtempSync(path.join(os.tmpdir(), 'docker-setup-buildx-')).split(path.sep).join(path.posix.sep);
910
jest.spyOn(context, 'tmpDir').mockImplementation((): string => {
@@ -90,6 +91,24 @@ describe('getCreateArgs', () => {
9091
'tls://foo:1234'
9192
]
9293
],
94+
[
95+
4,
96+
new Map<string, string>([
97+
['driver', 'remote'],
98+
['platforms', 'linux/arm64,linux/arm/v7'],
99+
['endpoint', 'tls://foo:1234'],
100+
['install', 'false'],
101+
['use', 'true'],
102+
]),
103+
[
104+
'create',
105+
'--name', 'builder-9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d',
106+
'--driver', 'remote',
107+
'--platform', 'linux/arm64,linux/arm/v7',
108+
'--use',
109+
'tls://foo:1234'
110+
]
111+
],
93112
])(
94113
'[%d] given %p as inputs, returns %p',
95114
async (num: number, inputs: Map<string, string>, expected: Array<string>) => {
@@ -103,6 +122,57 @@ describe('getCreateArgs', () => {
103122
);
104123
});
105124

125+
describe('getAppendArgs', () => {
126+
beforeEach(() => {
127+
process.env = Object.keys(process.env).reduce((object, key) => {
128+
if (!key.startsWith('INPUT_')) {
129+
object[key] = process.env[key];
130+
}
131+
return object;
132+
}, {});
133+
});
134+
135+
// prettier-ignore
136+
test.each([
137+
[
138+
0,
139+
new Map<string, string>([
140+
['install', 'false'],
141+
['use', 'true'],
142+
]),
143+
{
144+
"name": "aws_graviton2",
145+
"endpoint": "ssh://me@graviton2",
146+
"driver-opts": [
147+
"image=moby/buildkit:latest"
148+
],
149+
"buildkitd-flags": "--allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host",
150+
"platforms": "linux/arm64"
151+
},
152+
[
153+
'create',
154+
'--name', 'builder-9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d',
155+
'--append',
156+
'--node', 'aws_graviton2',
157+
'--driver-opt', 'image=moby/buildkit:latest',
158+
'--buildkitd-flags', '--allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host',
159+
'--platform', 'linux/arm64',
160+
'ssh://me@graviton2'
161+
]
162+
]
163+
])(
164+
'[%d] given %p as inputs, returns %p',
165+
async (num: number, inputs: Map<string, string>, node: nodes.Node, expected: Array<string>) => {
166+
inputs.forEach((value: string, name: string) => {
167+
setInput(name, value);
168+
});
169+
const inp = await context.getInputs();
170+
const res = await context.getAppendArgs(inp, node, '0.9.0');
171+
expect(res).toEqual(expected);
172+
}
173+
);
174+
});
175+
106176
describe('getInputList', () => {
107177
it('handles single line correctly', async () => {
108178
await setInput('foo', 'bar');

‎action.yml

+6
Original file line numberDiff line numberDiff line change
@@ -32,12 +32,18 @@ inputs:
3232
endpoint:
3333
description: 'Optional address for docker socket or context from `docker context ls`'
3434
required: false
35+
platforms:
36+
description: 'Fixed platforms for current node. If not empty, values take priority over the detected ones'
37+
required: false
3538
config:
3639
description: 'BuildKit config file'
3740
required: false
3841
config-inline:
3942
description: 'Inline BuildKit config'
4043
required: false
44+
append:
45+
description: 'Append additional nodes to the builder'
46+
required: false
4147

4248
outputs:
4349
name:

‎dist/index.js

+2-2
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎dist/index.js.map

+1-1
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎dist/licenses.txt

+50
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/advanced/append-nodes.md

+56
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# Append additional nodes to the builder
2+
3+
Buildx also supports running builds on multiple machines. This is useful for
4+
building [multi-platform images](https://docs.docker.com/build/building/multi-platform/)
5+
on native nodes for more complicated cases that are not handled by QEMU and
6+
generally have better performance or for distributing the build across multiple
7+
machines.
8+
9+
You can append nodes to the builder that is going to be created with the
10+
`append` input in the form of a YAML string document to remove limitations
11+
intrinsically linked to GitHub Actions (only string format is handled in the
12+
input fields):
13+
14+
| Name | Type | Description |
15+
|-------------------|--------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
16+
| `name` | String | [Name of the node](https://docs.docker.com/engine/reference/commandline/buildx_create/#node). If empty, it is the name of the builder it belongs to, with an index number suffix. This is useful to set it if you want to modify/remove a node in an underlying step of you workflow. |
17+
| `endpoint` | String | [Docker context or endpoint](https://docs.docker.com/engine/reference/commandline/buildx_create/#description) of the node to add to the builder |
18+
| `driver-opts` | List | List of additional [driver-specific options](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver-opt) |
19+
| `buildkitd-flags` | String | [Flags for buildkitd](https://docs.docker.com/engine/reference/commandline/buildx_create/#buildkitd-flags) daemon |
20+
| `platforms` | String | Fixed [platforms](https://docs.docker.com/engine/reference/commandline/buildx_create/#platform) for the node. If not empty, values take priority over the detected ones. |
21+
22+
Here is an example using remote nodes with the [`remote` driver](https://docs.docker.com/build/building/drivers/remote/)
23+
and [TLS authentication](auth.md#tls-authentication):
24+
25+
```yaml
26+
name: ci
27+
28+
on:
29+
push:
30+
31+
jobs:
32+
buildx:
33+
runs-on: ubuntu-latest
34+
steps:
35+
-
36+
name: Set up Docker Buildx
37+
uses: docker/setup-buildx-action@v2
38+
with:
39+
driver: remote
40+
endpoint: tcp://oneprovider:1234
41+
append: |
42+
- endpoint: tcp://graviton2:1234
43+
platforms: linux/arm64
44+
- endpoint: tcp://linuxone:1234
45+
platforms: linux/s390x
46+
env:
47+
BUILDER_NODE_0_AUTH_TLS_CACERT: ${{ secrets.ONEPROVIDER_CA }}
48+
BUILDER_NODE_0_AUTH_TLS_CERT: ${{ secrets.ONEPROVIDER_CERT }}
49+
BUILDER_NODE_0_AUTH_TLS_KEY: ${{ secrets.ONEPROVIDER_KEY }}
50+
BUILDER_NODE_1_AUTH_TLS_CACERT: ${{ secrets.GRAVITON2_CA }}
51+
BUILDER_NODE_1_AUTH_TLS_CERT: ${{ secrets.GRAVITON2_CERT }}
52+
BUILDER_NODE_1_AUTH_TLS_KEY: ${{ secrets.GRAVITON2_KEY }}
53+
BUILDER_NODE_2_AUTH_TLS_CACERT: ${{ secrets.LINUXONE_CA }}
54+
BUILDER_NODE_2_AUTH_TLS_CERT: ${{ secrets.LINUXONE_CERT }}
55+
BUILDER_NODE_2_AUTH_TLS_KEY: ${{ secrets.LINUXONE_KEY }}
56+
```

‎docs/advanced/auth.md

-5
Original file line numberDiff line numberDiff line change
@@ -41,11 +41,6 @@ the node in the list of nodes:
4141
* `BUILDER_NODE_<idx>_AUTH_TLS_CERT`
4242
* `BUILDER_NODE_<idx>_AUTH_TLS_KEY`
4343

44-
> **Note**
45-
>
46-
> The index is always `0` at the moment as we don't support (yet) appending new
47-
> nodes with this action.
48-
4944
```yaml
5045
name: ci
5146

‎jest.config.ts

+5-2
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,13 @@
11
module.exports = {
22
clearMocks: true,
33
moduleFileExtensions: ['js', 'ts'],
4-
setupFiles: ["dotenv/config"],
4+
setupFiles: ['dotenv/config'],
55
testMatch: ['**/*.test.ts'],
66
transform: {
77
'^.+\\.ts$': 'ts-jest'
88
},
9+
moduleNameMapper: {
10+
'^csv-parse/sync': '<rootDir>/node_modules/csv-parse/dist/cjs/sync.cjs'
11+
},
912
verbose: true
10-
}
13+
};

0 commit comments

Comments
 (0)