Skip to content

Commit df8cc86

Browse files
sdarwinsdarwin
committed
Document self-hosted runners
1 parent b1897a2 commit df8cc86

File tree

3 files changed

+278
-8
lines changed

3 files changed

+278
-8
lines changed

‎README.md

+139-8
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,147 @@
1+
# Github Actions Self-Hosted Runners
12

2-
### Introduction
3+
[The C++ Alliance](https://cppalliance.org/) is offering a service to Boost C++ repositories, enabling Github Actions workflows to be processed on AWS hosted runners.
34

4-
[GitHub Actions](https://github.com/features/actions) makes it easy to automate all your software workflows with world-class CI/CD. Build, test, and deploy your code right from GitHub. [The C++ Alliance](https://cppalliance.org/) has been working on a project to translate pre-existing .travis.yml files into GitHub Actions format for Boost Libraries.
5+
Often the standard Github runners are slow and unresponsive. Github throttles the number of simultaneous jobs. The monthly quota gets used up. By switching to self-hosted runners, CI can continue at a faster pace.
56

6-
### Instructions
7+
We have implemented a feature to globally switch the service ON or OFF. At the beginning of the month, or when Github is running smoothly, the jobs may be hosted on the normal Github runners. If things are moving slowly we will switch to the AWS machines. There is an opportunity to fine-tune the switch by adding logic that queries the Github API and makes intelligent decisions based on when the queues are busy.
78

8-
There are different options to consider.
9+
## Instructions
910

10-
1. If your .travis.yml has been kept reasonably up-to-date and/or contains customized tests, then the provided ci.yml is a convenient choice. It includes the same jobs as before. Merge the pull request which creates a .github/workflows/ci.yml file. Push another commit or pull request to trigger the CI tests. The same tests which passed on travis should be successful on GitHub Actions immediately.
11+
Modify your github actions workflow file, usually located at .github/workflows/ci.yml. A script has been provided in this repository [scripts/update_workflow.py](scripts/update_workflow.py) that will do most of the work if the ci.yml has a common format similar to boostorg/system. Otherwise, the updates can be made without using the script.
1112

12-
2. Another option, if your .travis.yml file is not current, could be to replace the entire workflow. In that case, do not merge the pull request. Copy a new template file from https://github.com/boostorg/boost-ci/blob/master/.github/workflows/ci.yml to your repository at .github/workflows/ci.yml. This may require debugging, it is not guaranteed to pass right away. The advantage of this choice is you will now have a relatively complete set of tests, if the previous .travis.yml file was not very extensive.
13+
LINUX:
1314

14-
3. Yet another option is a hybrid approach. Start with either of the files mentioned above. Copy-and-paste sections from the other file.
15+
Copy the script into your $PATH. For example:
1516

16-
If you have any questions or problems, please open an Issue in this github repo.
17+
```
18+
cp scripts/update_workflow.py /usr/local/bin/update_workflow.py
19+
```
20+
21+
Run the script:
22+
23+
```
24+
update_workflow.py .github/workflows/ci.yml
25+
```
26+
27+
WINDOWS:
28+
29+
It may be necessary on Windows to execute the script with the `python` executable:
30+
31+
```
32+
python update_workflow.py _target_file_
33+
```
34+
35+
In either case, it should create a .orig backup file. After running the script, check the updates, and fix anything that's missing.
36+
37+
```
38+
diff .github/workflows/ci.yml.orig .github/workflows/ci.yml
39+
```
40+
41+
These modifications should have been made in the workflow file:
42+
43+
- In the matrix section, quote the operating system labels. That is 'ubuntu-latest' or 'ubuntu-22.04'. Don't leave them as plain text ubuntu-latest. (The need for this change may depend on whether you have sections of json in the workflow or it is completely yaml format).
44+
45+
- Add a `runner-selection` job:
46+
47+
```
48+
jobs:
49+
runner-selection:
50+
# runs-on: ubuntu-latest
51+
runs-on: ${{ github.repository_owner == 'boostorg' && fromJSON('[ "self-hosted", "linux", "x64", "ubuntu-latest-aws" ]') || 'ubuntu-latest' }}
52+
outputs:
53+
labelmatrix: ${{ steps.aws_hosted_runners.outputs.labelmatrix }}
54+
steps:
55+
- name: AWS Hosted Runners
56+
id: aws_hosted_runners
57+
uses: cppalliance/[email protected]
58+
```
59+
60+
The initial runner-selection job may run on either a self-hosted or GitHub host. There are pros and cons to each. Consider both options and set the one you prefer. If GitHub is being responsive and everything is pointed to using GitHub, a self-hosted runner at the first step causes a delay. If GitHub is slow and everything is pointed to AWS, then it's faster to use self-hosted.
61+
62+
- Set the `runs-on` and `needs` configuration in all jobs:
63+
64+
```
65+
needs: [runner-selection]
66+
runs-on: ${{ fromJSON(needs.runner-selection.outputs.labelmatrix)[matrix.os] }}
67+
```
68+
69+
In the above example, notice where it says "matrix.os". That is whatever the runner label had been before.
70+
71+
If a job specified the exact label 'ubuntu-22.04', then the section should be:
72+
73+
```
74+
needs: [runner-selection]
75+
runs-on: ${{ fromJSON(needs.runner-selection.outputs.labelmatrix)['ubuntu-22.04'] }}
76+
```
77+
78+
After `update_workflow.py` has completed you may modify ci.yml manually to adjust anything.
79+
80+
If specific jobs should be excluded add '-no-aws' to the label, such as 'ubuntu-latest-no-aws'. `runner-selection` will resolve that back to 'ubuntu-latest'.
81+
82+
## Supported Runners
83+
84+
The list of runner images and installed packages may be found at https://github.com/cppalliance/terraform-aws-github-runner/blob/cppal/images in the *-cppal files.
85+
86+
| AMI Images |
87+
| ---------- |
88+
| ubuntu-bionic-arm64-cppal |
89+
| ubuntu-bionic-cppal |
90+
| ubuntu-focal-arm64-cppal |
91+
| ubuntu-focal-cppal |
92+
| ubuntu-jammy-arm64-cppal |
93+
| ubuntu-jammy-cppal |
94+
| windows-2019-cppal |
95+
| windows-2022-cppal |
96+
97+
Linux:
98+
99+
| Runner Label |
100+
| ------------- |
101+
| ubuntu-20.04 |
102+
| ubuntu-22.04, ubuntu-latest |
103+
104+
Windows:
105+
106+
| Runner Label | Visual Studio |
107+
| ------------- | ------------- |
108+
| windows-2019 | msvc-14.0 |
109+
| windows-2019 | msvc-14.1 |
110+
| windows-2019 | msvc-14.2 |
111+
| windows-2022, windows-latest | msvc-14.3 |
112+
113+
Mac:
114+
115+
| Runner Label |
116+
| ------------- |
117+
| macos-11 |
118+
| macos-12, macos-latest |
119+
120+
MacOS is not being hosted currently. Those jobs will continue to use Github runners instead of AWS.
121+
122+
## GitHub App
123+
124+
A "GitHub App" called "Terraform-AWS-Runners-Boost" should already be installed at the Organization level. However, it must also be installed to each repository that will use self-hosted runners. Contact an organization-level administrator so they can modify the install settings of the GitHub App and include the repository. Admin instructions are covered in [docs/admin-guide.md](docs/admin-guide.md)
125+
126+
## Advanced Configuration
127+
128+
### Temporarily disable self-hosted runners
129+
130+
To ensure self-hosted runners are not used, modify the runner selection:
131+
- `run-on` a GitHub runner
132+
- self_hosted_runners_override set to 'false'
133+
134+
```
135+
jobs:
136+
runner-selection:
137+
runs-on: ubuntu-latest
138+
# runs-on: [ self-hosted, linux, x64, ubuntu-latest-aws ]
139+
outputs:
140+
labelmatrix: ${{ steps.aws_hosted_runners.outputs.labelmatrix }}
141+
steps:
142+
- name: AWS Hosted Runners
143+
id: aws_hosted_runners
144+
uses: cppalliance/[email protected]
145+
with:
146+
self_hosted_runners_override: 'false'
147+
```

‎docs/admin-guide.md

+47
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
2+
## Admin Instructions
3+
4+
The project is based on "Terraform module for scalable self hosted GitHub action runners" https://github.com/philips-labs/terraform-aws-github-runner . CPPAlliance is maintaining a fork of that repo with updated AMI images and runner configurations at https://github.com/cppalliance/terraform-aws-github-runner
5+
6+
### Create a GitHub App
7+
8+
This is a one-time action. Replace "boostorg" in the following steps if another organization is being used.
9+
10+
Go to the GitHub Apps setting page https://github.com/organizations/boostorg/settings/apps (This page may also be reached from Settings https://github.com/organizations/boostorg/settings and then navigate to Developer Settings -> GitHub Apps)
11+
12+
In the Management section, add other admin users. Most steps can be done by a delegated user, except the final installation.
13+
14+
Click "New GitHub App" to create a new GitHub App.
15+
16+
On the "General" page of the app:
17+
18+
Name: Terraform-AWS-Runners-Boost
19+
Description: Self-hosted github actions runners
20+
Homepage URL: https://github.com/cppalliance/githubactions
21+
22+
Webhook:
23+
Active: Check this box
24+
Webhook URL: fill in the value from Terraform
25+
Webhook secret: fill in the value from Terraform
26+
27+
Under Private Keys click "Generate a private key". Download a private key. The key and the "App ID:" at the top of the page will be entered in Terraform.
28+
29+
Click "Save Changes"
30+
31+
On the "Permissions and Events" page of the app, under Repository Permissions:
32+
Actions: Read-Only
33+
Administration: Read and Write
34+
Checks: Read-Only
35+
Metadata: Read-Only
36+
37+
Subscribe to Events:
38+
choose Workflow Job
39+
40+
Click "Save Changes"
41+
42+
## Install the App
43+
44+
In the "Install App" page of the app configuration, choose the Organization and then the target repositories. This step must be done by an organization administrator.
45+
46+
Only apply the app to selected repositories, not the entire organization. Whenever a repository will add self-hosted runners the organization administrator must modify the installation setting.
47+

‎scripts/update_workflow.py

+92
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
#!/usr/bin/python3
2+
3+
#
4+
# update_workflow.py
5+
#
6+
# Modifies github actions workflows to use self-hosted runners.
7+
#
8+
# python3 update_workflow.py __path_to_ci_file__
9+
#
10+
11+
import sys
12+
import re
13+
import shutil
14+
15+
targetfile=sys.argv[1]
16+
17+
shutil.copyfile(targetfile, targetfile + '.orig')
18+
19+
with open(targetfile, 'r') as file:
20+
data = file.read()
21+
22+
# Add single quotes around operating system names if they are missing. 'ubuntu-latest'
23+
24+
data=re.sub(
25+
pattern=r'([^"\'])(\S*)-latest([^"\'])',
26+
repl='\\1\'\\2-latest\'\\3',
27+
string=data
28+
)
29+
30+
data=re.sub(
31+
pattern=r'([^"\'])ubuntu-(\d\d)\.(\d\d)([^"\'])',
32+
repl='\\1\'ubuntu-\\2.\\3\'\\4',
33+
string=data
34+
)
35+
36+
data=re.sub(
37+
pattern=r'([^"\'])windows-(\d{4})([^"\'])',
38+
repl='\\1\'windows-\\2\'\\3',
39+
string=data
40+
)
41+
42+
data=re.sub(
43+
pattern=r'([^"\'])macos-(\d{2})([^"\'])',
44+
repl='\\1\'macos-\\2\'\\3',
45+
string=data
46+
)
47+
48+
data=re.sub(
49+
pattern=r'([^\S\r\n]*)runs-on: \${{\s*matrix.os\s*}}',
50+
repl='\\1needs: [runner-selection]\n\\1runs-on: ${{ fromJSON(needs.runner-selection.outputs.labelmatrix)[matrix.os] }}',
51+
string=data
52+
)
53+
54+
data=re.sub(
55+
pattern=r"([^\S\r\n]*)runs-on:(\s*)('ubuntu.*').*",
56+
repl='\\1needs: [runner-selection]\n\\1runs-on: ${{ fromJSON(needs.runner-selection.outputs.labelmatrix)[\\3] }}',
57+
string=data
58+
)
59+
60+
data=re.sub(
61+
pattern=r"([^\S\r\n]*)runs-on:(\s*)('windows.*').*",
62+
repl='\\1needs: [runner-selection]\n\\1runs-on: ${{ fromJSON(needs.runner-selection.outputs.labelmatrix)[\\3] }}',
63+
string=data
64+
)
65+
66+
data=re.sub(
67+
pattern=r"([^\S\r\n]*)runs-on:(\s*)('mac.*').*",
68+
repl='\\1needs: [runner-selection]\n\\1runs-on: ${{ fromJSON(needs.runner-selection.outputs.labelmatrix)[\\3] }}',
69+
string=data
70+
)
71+
72+
job_to_include="""
73+
runner-selection:
74+
# runs-on: ubuntu-latest
75+
runs-on: ${{ github.repository_owner == 'boostorg' && fromJSON('[ "self-hosted", "linux", "x64", "ubuntu-latest-aws" ]') || 'ubuntu-latest' }}
76+
outputs:
77+
labelmatrix: ${{ steps.aws_hosted_runners.outputs.labelmatrix }}
78+
steps:
79+
- name: AWS Hosted Runners
80+
id: aws_hosted_runners
81+
uses: cppalliance/[email protected]
82+
"""
83+
84+
data=re.sub(
85+
pattern=r'(jobs:)',
86+
repl='\\1' + job_to_include,
87+
string=data
88+
)
89+
90+
f = open(targetfile, "w")
91+
f.write(data)
92+
f.close()

0 commit comments

Comments
 (0)