Skip to content

Commit 63f634a

Browse files
authored
add vault agent documentation (#49)
1 parent 7ea0a62 commit 63f634a

File tree

3 files changed

+202
-34
lines changed

3 files changed

+202
-34
lines changed

Diff for: .editorconfig

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
root = true
2+
3+
[*]
4+
charset = utf-8
5+
insert_final_newline = true
6+
trim_trailing_whitespace = true
7+
indent_style = space
8+
9+
[*.{java,groovy}]
10+
indent_size = 4
11+
12+
[*.{xml,html,jelly}]
13+
indent_size = 2
14+
15+
[*.md]
16+
trim_trailing_whitespace = false

Diff for: README.md

+184-32
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
This plugin adds a build wrapper to set environment variables from a HashiCorp [Vault](https://www.vaultproject.io/) secret. Secrets are generally masked in the build log, so you can't accidentally print them.
55
It also has the ability to inject Vault credentials into a build pipeline or freestyle job for fine-grained vault interactions.
66

7-
# Vault Authentication Backends
7+
## Vault Authentication Backends
88
This plugin allows authenticating against Vault using the AppRole authentication backend. Hashicorp recommends using AppRole for Servers / automated workflows (like Jenkins) and using Tokens (default mechanism, Github Token, ...) for every developer's machine.
99
Furthermore, this plugin allows using a Github personal access token, or a Vault Token - either configured directly in Jenkins or read from an arbitrary file on the Jenkins Machine.
1010

@@ -28,7 +28,7 @@ The `Descriptor` needs to be annotated with `@Extension`. Your credential needs
2828
See [VaultAppRoleCredential.java](https://github.com/jenkinsci/hashicorp-vault-plugin/blob/master/src/main/java/com/datapipe/jenkins/vault/credentials/VaultAppRoleCredential.java) for an example.
2929

3030

31-
# Plugin Usage
31+
## Plugin Usage
3232
### Configuration
3333
You can configure the plugin on three different levels:
3434
* Global: in your global config
@@ -105,7 +105,7 @@ node {
105105
// optional configuration, if you do not provide this the next higher configuration
106106
// (e.g. folder or global) will be used
107107
def configuration = [vaultUrl: 'http://my-very-other-vault-url.com',
108-
vaultCredentialId: 'my-vault-cred-id'
108+
vaultCredentialId: 'my-vault-cred-id',
109109
engineVersion: 1]
110110
// inside this block your credentials will be available as env variables
111111
withVault([configuration: configuration, vaultSecrets: secrets]) {
@@ -117,7 +117,7 @@ node {
117117
```
118118
In the future we might migrate to a [BuildStep](http://javadoc.jenkins-ci.org/hudson/tasks/BuildStep.html) instead of a BuildWrapper.
119119

120-
# Inject Vault Credentials into your Job
120+
## Inject Vault Credentials into your Job
121121
### Pipeline Usage
122122
![withCredentials Block](docs/images/pipeline_withCredentials.png)
123123

@@ -137,35 +137,187 @@ node {
137137
### FreeStyle Job
138138
![freeStyle Credential Binding](docs/images/freestyle_injectCreds.png)
139139

140+
## Configuration as Code
141+
There is an easier way to setup the global Vault configuration on your Jenkins server.
142+
No need for messing around in the UI.
140143

141-
# Migration Guide
142-
143-
### Upgrade from 1.x to 2.0
144-
The `BuildWrapper` did not change, so no changes to your Jenkinsfile should be necessary. However, you need to reconfigure Vault in your Jenkins instance based on the instructions above. There is no way to smoothly upgrade this, because this is a major rewrite and handling of configuration completly changed.
145-
146-
# CHANGELOG
147-
* **2018/08/22** - Feature Release - 2.2.0
148-
* Add support for GCP authentication
149-
* **2018/05/01** - Bugfix Release - 2.1.1
150-
* MaskingConsoleLogFilter should filter out null secrets [JENKINS-46792](https://issues.jenkins-ci.org/browse/JENKINS-38647)
151-
* [Avoid NPE Crash](https://github.com/jenkinsci/hashicorp-vault-plugin/pull/13)
152-
* Switch to SimpleBuildWrapper for pipeline compatibility [JENKINS-48049](https://issues.jenkins-ci.org/browse/JENKINS-48049)
153-
* Dynamic secrets should be revoked after build wrapper completes [JENKINS-46794](https://issues.jenkins-ci.org/browse/JENKINS-46794)
154-
* **2017/05/22** - Feature Release - 2.1.0
155-
* Vault Key Not Saved In Vault Error Messaging [JENKINS-38647](https://issues.jenkins-ci.org/browse/JENKINS-38647)
156-
* Add support github token auth [JENKINS-38939](https://issues.jenkins-ci.org/browse/JENKINS-38939)
157-
* **2017/05/19** - Bugfix Release - 2.0.1
158-
* Build fails if plugin is enabled for a job without secrets specified [JENKINS-44163](https://issues.jenkins-ci.org/browse/JENKINS-441630)
159-
* **2017/04/27** - Breaking change release (AppRole auth backend, Folder ability, improved configuration, ...)
160-
* **2017/04/10** - Feature Release - 1.4
161-
* Support reading Vault Token from file on disk [JENKINS-37713](issues.jenkins-ci.org/browse/JENKINS-37713)
162-
* Using credentials plugin for authentication token [JENKINS-38646](issues.jenkins-ci.org/browse/JENKINS-38646)
163-
* **2017/03/03** - Feature Release - 1.3
164-
* Vault Plugin should mask credentials in build log [JENKINS-39383](issues.jenkins-ci.org/browse/JENKINS-39383)
165-
* **2016/08/15** - Re-release due to failed maven release - 1.2
166-
* **2016/08/11** - Bugfix release - 1.1
167-
* Refactor to allow getting multiple vault keys in a single API call [JENKINS-37151](https://issues.jenkins-ci.org/browse/JENKINS-37151)
168-
* **2016/08/02** - Initial release - 1.0
144+
Jenkins Configuration as Code often shorten to [JCasC] or simplify [Configuration as Code plugin]
145+
allows you to configure Jenkins via a yaml file. If you are a first time user, you can learn more about [JCasC] :point_left:
146+
147+
Hashicorp Plugin also adds an extension to [JCasC] by providing a Secret Source for [Configuration as Code plugin] to read secrets from, which you can [read about here](#hashicorp-vault-plugin-as-a-secret-source-for-jcasc)
148+
149+
### Prerequisite:
150+
151+
Install `Configuration as Code` Plugin on your Jenkins instance.
152+
153+
Refer to [Installing a new plugin in Jenkins](https://jenkins.io/doc/book/managing/plugins/#installing-a-plugin).
154+
155+
#### Add configuration YAML:
156+
157+
There are multiple ways to load JCasC yaml file to configure Jenkins:
158+
159+
* JCasC by default searches for a file with the name `jenkins.yaml` in `$JENKINS_ROOT`.
160+
161+
* The JCasC looks for an environment variable `CASC_JENKINS_CONFIG` which contains the path for the configuration `yaml` file.
162+
163+
* A path to a folder containing a set of config files e.g. `/var/jenkins_home/casc_configs`.
164+
165+
* A full path to a single file e.g. `/var/jenkins_home/casc_configs/jenkins.yaml`.
166+
167+
* A URL pointing to a file served on the web e.g. `https://<your-domain>/jenkins.yaml`.
168+
169+
* You can also set the configuration yaml path in the UI. Go to `<your-jenkins-domain>/configuration-as-code`. Enter path or URL to `jenkins.yaml` and select `Apply New Configuration`.
170+
171+
To configure your Vault in Jenkins add the following to `jenkins.yaml`:
172+
173+
```yaml
174+
unclassified:
175+
hashicorpVault:
176+
configuration:
177+
vaultCredentialId: "vaultToken"
178+
vaultUrl: "${CASC_VAULT_AGENT_ADDR:-http://localhost:8100}" # Vault Agent Addr
179+
180+
credentials:
181+
system:
182+
domainCredentials:
183+
- credentials:
184+
- vaultTokenCredential:
185+
description: "Uber Token"
186+
id: "vaultToken"
187+
scope: GLOBAL
188+
token: "${MY_SECRET_TOKEN}"
189+
```
190+
191+
See [handling secrets section](https://github.com/jenkinsci/configuration-as-code-plugin#handling-secrets) in JCasC documentation for better security.
192+
193+
You can also configure `VaultGithubTokenCredential`, or `VautGCPCredential` or `VaultAppRoleCredential`
194+
195+
If you are unsure about how to do it from `yaml`. You can still use the UI to configure credentials.
196+
After you configured Credentials and the Global Vault configuration.
197+
you can use the export feature build into JCasC by visiting `<your-jenkins-domain>/configuration-as-code/viewExport`
198+
199+
### HashiCorp Vault Plugin as a Secret Source for JCasC
200+
201+
We can provide these initial secrets for JCasC
202+
The secret source for JCasC is configured via environment variables as way to get access to vault at startup and when configuring Jenkins instance.
203+
204+
[For Security and compatibility considerations please read more here](https://github.com/jenkinsci/configuration-as-code-plugin#security-and-compatibility-considerations)
205+
206+
- The environment variable `CASC_VAULT_PW` must be present, if token is not used and appRole/Secret is not used. (Vault password.)
207+
- The environment variable `CASC_VAULT_USER` must be present, if token is not used and appRole/Secret is not used. (Vault username.)
208+
- The environment variable `CASC_VAULT_APPROLE` must be present, if token is not used and U/P not used. (Vault AppRole ID.)
209+
- The environment variable `CASC_VAULT_APPROLE_SECRET` must be present, it token is not used and U/P not used. (Vault AppRole Secret ID.)
210+
- The environment variable `CASC_VAULT_TOKEN` must be present, if U/P is not used. (Vault token.)
211+
- The environment variable `CASC_VAULT_PATHS` must be present. (Comma separated vault key paths. For example, `secret/jenkins,secret/admin`.)
212+
- The environment variable `CASC_VAULT_URL` must be present. (Vault url, including port number.)
213+
- The environment variable `CASC_VAULT_AGENT_ADDR` is optional. It takes precedence over `CASC_VAULT_URL` and is used for connecting to a Vault Agent. [See this section](#vault-agent)
214+
- The environment variable `CASC_VAULT_MOUNT` is optional. (Vault auth mount. For example, `ldap` or another username & password authentication type, defaults to `userpass`.)
215+
- The environment variable `CASC_VAULT_NAMESPACE` is optional. If used, sets the Vault namespace for Enterprise Vaults.
216+
- The environment variable `CASC_VAULT_FILE` is optional, provides a way for the other variables to be read from a file instead of environment variables.
217+
- The environment variable `CASC_VAULT_ENGINE_VERSION` is optional. If unset, your vault path is assumed to be using kv version 2. If your vault path uses engine version 1, set this variable to `1`.
218+
- The issued token should have read access to vault path `auth/token/lookup-self` in order to determine its expiration time. JCasC will re-issue a token if its expiration is reached (except for `CASC_VAULT_TOKEN`).
219+
220+
If the environment variables `CASC_VAULT_URL` and `CASC_VAULT_PATHS` are present, JCasC will try to gather initial secrets from Vault. However for it to work properly there is a need for authentication by either the combination of `CASC_VAULT_USER` and `CASC_VAULT_PW`, a `CASC_VAULT_TOKEN`, or the combination of `CASC_VAULT_APPROLE` and `CASC_VAULT_APPROLE_SECRET`. The authenticated user must have at least read access.
221+
222+
You can also provide a `CASC_VAULT_FILE` environment variable where you load the secrets from a file.
223+
224+
File should be in a Java Properties format
225+
226+
```properties
227+
CASC_VAULT_PW=PASSWORD
228+
CASC_VAULT_USER=USER
229+
CASC_VAULT_TOKEN=TOKEN
230+
CASC_VAULT_PATHS=secret/jenkins/master,secret/admin
231+
CASC_VAULT_URL=https://vault.dot.com
232+
CASC_VAULT_MOUNT=ldap
233+
```
234+
235+
A good use for `CASC_VAULT_FILE` would be together with docker secrets.
236+
237+
```yaml
238+
version: "3.6"
239+
240+
services:
241+
jenkins:
242+
environment:
243+
CASC_VAULT_FILE: /run/secrets/jcasc_vault
244+
restart: always
245+
build: .
246+
image: jenkins.master:v1.0
247+
ports:
248+
- 8080:8080
249+
- 50000:50000
250+
volumes:
251+
- jenkins-home:/var/jenkins_home
252+
secrets:
253+
- jcasc_vault
254+
255+
volumes:
256+
jenkins-home:
257+
258+
secrets:
259+
jcasc_vault:
260+
file: ./secrets/jcasc_vault
261+
```
262+
263+
### Vault Agent
264+
265+
[For the use-case of Vault Agent read here](https://www.vaultproject.io/docs/agent/)
266+
267+
When `CASC_VAULT_AGENT_ADDR` is specified, you only need to specify `CASC_VAULT_PATHS` and optionally `CASC_VAULT_ENGINE_VERSION`
268+
Since Vault Agent must be configured to handle auto authentication.
269+
270+
Here is an example of how to configure your Vault Agent with an app role.
271+
[Vault Agent supports multiple auto-auth methods](https://www.vaultproject.io/docs/agent/autoauth/methods/index.html)
272+
273+
```hcl
274+
pid_file = "/tmp/agent_pidfile"
275+
auto_auth {
276+
method {
277+
type = "approle"
278+
config = {
279+
role_id_file_path = "/home/vault/role_id"
280+
secret_id_file_path = "/home/vault/secret_id"
281+
}
282+
}
283+
sink {
284+
type = "file"
285+
config = {
286+
path = "/tmp/file-foo"
287+
}
288+
}
289+
}
290+
cache {
291+
use_auto_auth_token = true
292+
}
293+
listener "tcp" {
294+
address = "0.0.0.0:8200"
295+
tls_disable = true
296+
}
297+
```
298+
299+
Ideally your Vault Agent should be running on the same Machine or running as a Container networked together.
300+
You ought to block any connection to Vault Agent for anything that is not considered localhost.
301+
302+
For setup with Jenkins and Vault Agent running on the same host you can achieve this by using
303+
```hcl
304+
listener "tcp" {
305+
address = "127.0.0.1:8200"
306+
tls_disable = true
307+
}
308+
```
309+
310+
For Containers you would need to use listener address of `0.0.0.0:8200`.
311+
You should ___never expose___ the Vault Agent port. You ___OUGHT___ to network Vault Agent container and Jenkins container together.
312+
313+
```hcl
314+
listener "tcp" {
315+
address = "0.0.0.0:8200"
316+
tls_disable = true
317+
}
318+
```
169319

170320
[global_configuration]: docs/images/global_configuration.png
171321
[job_configuration]: docs/images/job_configuration.png
322+
[JCasC]: https://github.com/casz/configuration-as-code-plugin
323+
[Configuration as Code plugin]: https://github.com/casz/configuration-as-code-plugin

Diff for: pom.xml

+2-2
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,8 @@
2020
<configuration-as-code.version>1.29</configuration-as-code.version>
2121
</properties>
2222
<name>HashiCorp Vault Plugin</name>
23-
<description>Build Wrapper for reading secrets in a HashiCorp Vault</description>
24-
<url>https://wiki.jenkins-ci.org/display/JENKINS/HashiCorp+Vault+Plugin</url>
23+
<description>Reading secrets from HashiCorp Vault for a Pipeline or as a Secret Source for JCasC</description>
24+
<url>https://github.com/jenkinsci/hashicorp-vault-plugin</url>
2525
<organization>
2626
<name>Datapipe, Inc.</name>
2727
<url>https://www.datapipe.com</url>

0 commit comments

Comments
 (0)