Skip to content

Commit e635c0c

Browse files
author
Florian Medja
committed
Add documentation for building Logstash custom image with opensearch plugin
Signed-off-by: Florian Medja <[email protected]>
1 parent d60a58f commit e635c0c

File tree

2 files changed

+83
-0
lines changed

2 files changed

+83
-0
lines changed

DEVELOPER_GUIDE.md

+4
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
- [Configuration for Logstash Output OpenSearch Plugin](#configuration-for-logstash-output-opensearch-plugin)
1010
- [Submitting Changes](#submitting-changes)
1111
- [Backports](#backports)
12+
- [Building Custom Docker Images](docs/building_custom_docker_images.md)
1213

1314
# Developer Guide
1415

@@ -215,3 +216,6 @@ original PR with an appropriate label `backport <backport-branch-name>` is merge
215216
run successfully on the PR. For example, if a PR on main needs to be backported to `1.x` branch, add a label
216217
`backport 1.x` to the PR and make sure the backport workflow runs on the PR along with other checks. Once this PR is
217218
merged to main, the workflow will create a backport PR to the `1.x` branch.
219+
220+
# [Building Custom Docker Images](docs/building_custom_docker_images.md)
221+

docs/building_custom_docker_images.md

+79
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
- [Building Custom Docker Images](#building-custom-docker-images)
2+
- [Logstash 8.x](#logstash-8x)
3+
- [Logstash 7.x](#logstash-7x)
4+
- [Build Logstash Output OpenSearch Plugin Gem](#build-logstash-output-opensearch-plugin-gem)
5+
- [Dockerfile](#dockerfile)
6+
7+
8+
# Building Custom Docker Images
9+
10+
To build an image that is not available in the [official Docker repository tags](https://hub.docker.com/r/opensearchproject/logstash-oss-with-opensearch-output-plugin/tags), or to add specific plugins to your image, you can create a custom Dockerfile.
11+
12+
The process varies depending on whether you want to build an image with `Logstash 8.x` versions or `Logstash 7.x` versions
13+
14+
## Logstash 8.x
15+
16+
Create this `Dockerfile` to build an image with `OpenSearch 2.0.2` for **`Logstash 8.x`**
17+
18+
``` Dockerfile
19+
ARG APP_VERSION
20+
21+
FROM docker.elastic.co/logstash/logstash-oss:${APP_VERSION}
22+
RUN logstash-plugin install --version 7.1.1 logstash-integration-aws
23+
RUN logstash-plugin install --version 2.0.2 logstash-output-opensearch
24+
```
25+
26+
## Logstash 7.x
27+
28+
For **`Logstash 7.x`** , the Logstash output OpenSearch gem needs to be build.
29+
30+
### Build Logstash Output OpenSearch Plugin Gem
31+
32+
1. Clone `logstash-output-opensearch repo`
33+
34+
```sh
35+
git clone https://github.com/opensearch-project/logstash-output-opensearch.git
36+
```
37+
38+
2. Checkout the tag for the plugin version you want to build. For the version [2.0.2](https://github.com/opensearch-project/logstash-output-opensearch/tree/2.0.2) for example
39+
40+
```sh
41+
git checkout 2.0.2
42+
```
43+
44+
45+
3. Remove [this line that adds the json version spec](https://github.com/opensearch-project/logstash-output-opensearch/blob/2.0.2/logstash-output-opensearch.gemspec#L49). This version of the JSON gem is incompatible with `Logstash version 7.x`.
46+
47+
4. Build the gem by running the following command:
48+
49+
```sh
50+
gem build logstash-output-opensearch.gemspec
51+
```
52+
53+
The Gemfile `logstash-output-opensearch-2.0.2-x86_64-linux.gem` will be generated.
54+
55+
### Dockerfile
56+
57+
Create this Dockerfile to build an image with `logstash version 7.x` and the previously generated `Gemfile` :
58+
59+
```Dockerfile
60+
ARG APP_VERSION
61+
62+
FROM docker.elastic.co/logstash/logstash-oss:${APP_VERSION}
63+
64+
USER logstash
65+
# Remove existing logstash aws plugins and install logstash-integration-aws to keep sdk dependency the same
66+
# https://github.com/logstash-plugins/logstash-mixin-aws/issues/38
67+
# https://github.com/opensearch-project/logstash-output-opensearch#configuration-for-logstash-output-opensearch-plugin
68+
RUN logstash-plugin remove logstash-input-s3
69+
RUN logstash-plugin remove logstash-input-sqs
70+
RUN logstash-plugin remove logstash-output-s3
71+
RUN logstash-plugin remove logstash-output-sns
72+
RUN logstash-plugin remove logstash-output-sqs
73+
RUN logstash-plugin remove logstash-output-cloudwatch
74+
75+
RUN logstash-plugin install --version 7.1.1 logstash-integration-aws
76+
77+
COPY logstash-output-opensearch-2.0.2-x86_64-linux.gem /usr/share
78+
RUN logstash-plugin install /usr/share/logstash-output-opensearch-2.0.2-x86_64-linux.gem
79+
```

0 commit comments

Comments
 (0)