Skip to content

Commit 477c1aa

Browse files
Stephen Taylorsshanks-kx
Stephen Taylor
and
sshanks-kx
authored
DRAFT docs imported prior to removal from code.kx.com (#16)
* imported MDs from code.kx.com * fixed links to documentation * translated MDs to GFM; fixed links * minor edits * fixed link * fixed link to examples dir --------- Co-authored-by: Simon Shanks <[email protected]>
1 parent b2a0af1 commit 477c1aa

8 files changed

+570
-32
lines changed

‎README.md

+17-15
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,27 @@
1-
# Prometheus Exporter for kdb+
1+
# ![Prometheus Exporter](prometheus.png) Prometheus Exporter for kdb+
22

33
[![GitHub release (latest by date)](https://img.shields.io/github/v/release/kxsystems/prometheus-kdb-exporter?include_prereleases)](https://github.com/kxsystems/prometheus-kdb-exporter/releases)
44

5-
## Introduction
5+
66

77
This interface provides a method by which to expose metrics from a kdb+ process or multiple processes to Prometheus for monitoring. This is done via the script `q/exporter.q` which exposes kdb+ process metrics which can be consumed by Prometheus.
88

9-
This interface is part of the [_Fusion for kdb+_](https://code.kx.com/v2/interfaces/fusion/) project.
9+
This interface is part of the [_Fusion for kdb+_](https://code.kx.com/q/interfaces#fusion/) project.
1010

1111
## New to kdb+ ?
1212

13-
Kdb+ is the world's fastest time-series database, optimized for ingesting, analyzing and storing massive amounts of structured data. To get started with kdb+, please visit https://code.kx.com/q/learn/ for downloads and developer information. For general information, visit https://kx.com/
13+
Kdb+ is the world's fastest time-series database, optimized for ingesting, analyzing and storing massive amounts of structured data. To get started with kdb+, visit https://code.kx.com/q/learn/ for downloads and developer information. For general information, visit https://kx.com/
1414

1515
## What is Prometheus ?
1616

17-
Prometheus is an open source monitoring solution which facilitates metrics gathering, querying and alerting for a wealth of different 3rd party languages and applications. It also provides integration with Kubernetes for automatic discovery of supported applications.
17+
Prometheus is an open source monitoring solution which facilitates metrics gathering, querying and alerting for a wealth of different 3rd-party languages and applications. It also provides integration with Kubernetes for automatic discovery of supported applications.
1818

19-
Visualization and querying can be done through the Prometheus built in expression browser, or more commonly via Grafana. An example of this using docker is provided with this interface [here](./example/).
19+
Visualization and querying can be done through the Prometheus built in expression browser, or more commonly via Grafana.
20+
The repo includes [an example](examples) of this using Docker.
2021

21-
## Quick Start
22+
## Quick start
2223

23-
Install the appropriate q scripts to `$QHOME`/`%QHOME%` as appropriate using the `install.sh`/`install.bat` files
24+
Install the appropriate q scripts to `$QHOME`/`%QHOME%` using the `install.sh`/`install.bat` files
2425

2526
```bash
2627
## Linux/MacOS
@@ -32,25 +33,26 @@ install.bat
3233

3334
Run kdb+ with the supplied q script. This script will expose metrics on port 8080 which can be monitored by Prometheus
3435

35-
```
36+
```bash
3637
q q/exporter.q -p 8080
3738
```
3839

39-
Once running, you can use your web browser to view the currently exposed statistics on the metrics URL e.g. http://localhost:8080/metrics. The metrics exposed will be the metric values at the time at which the URL is requested.
40+
Once running, use your web browser to view the currently exposed statistics on the metrics URL e.g. http://localhost:8080/metrics. The metrics exposed will be the metric values at the time at which the URL is requested.
4041

41-
## Unsupported Functionality
42+
## Unsupported functionality
4243

43-
* This interface does not provide service discovery. Prometheus itself has support for multiple mechanisms such as DNS, Kubernetes, EC2, file based config, etc in order to discover all the kdb+ instances within your environment.
44+
This interface does not provide service discovery. Prometheus itself has support for multiple mechanisms such as DNS, Kubernetes, EC2, file based config, etc., to discover all the kdb+ instances within your environment.
4445

4546

4647
## Documentation
4748

48-
Extensive documentation for this interface is available on the code.kx.com website [here](https://code.kx.com/q/interfaces/prom/exporter/).
49+
:open_file_folder: [`docs`](docs)
50+
4951

5052
## Status
5153

5254
The prometheus-kdb-exporter interface is provided here under an Apache 2.0 license.
5355

54-
If you find issues with the interface or have feature requests please consider raising an issue [here](https://github.com/KxSystems/prometheus-kdb-exporter/issues).
56+
If you find issues with the interface or have feature requests please [raise an issue](../../issues).
5557

56-
If you wish to contribute to this project please follow the contributing guide [here](CONTRIBUTING.md).
58+
To contribute to this project, please follow the [contribution guide](CONTRIBUTING.md).

‎docs/README.md

+69
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# ![Prometheus](../prometheus.png) Prometheus Exporter
2+
3+
4+
[Prometheus](https://prometheus.io/docs/instrumenting/exporters/) is free software which facilitates metric gathering, querying and alerting for a wealth of different third-party languages and applications. It also provides integration with Kubernetes for automatic discovery of supported applications.
5+
6+
Visualization and querying can be done through its built-in expression browser or, more commonly, via [Grafana](https://grafana.com/).
7+
8+
An environment being administered or analyzed by Prometheus can include current and past metrics exposed by kdb+.
9+
10+
11+
## Use cases
12+
13+
The following are potential use cases for the interface. This is by no means an exhaustive list.
14+
15+
- effects from version upgrades (e.g. performance before/after changes)
16+
- alerts when your that a licence may be due to expire
17+
- bad use of symbol types within an instance
18+
19+
20+
## Kdb+/Prometheus-Exporter integration
21+
22+
This interface
23+
24+
- provides a script for useful general metrics that can be extended if required
25+
- allows correlations between different instances, metrics, exporters and installs to be easily identified
26+
27+
Some caveats regarding where this interface in its current iteration can be used
28+
29+
- This interface does not provide service discovery. Prometheus itself has support for multiple mechanisms such as DNS, Kubernetes, EC2, file based config, etc., to discover all the kdb+ instances within your environment.
30+
- You may need to extend this script to provide more relevant metrics for your environment. Please consider contributing if your change may be generic enough to have a wider user benefit.
31+
- General machine/Kubernetes/cloud metrics on which kdb+ is running. Metrics can be gathered by such exporters as the node exporter. Metrics from multiple exporters can be correlated to provide a bigger picture of your environment conditions.
32+
33+
34+
## Metrics
35+
36+
In Prometheus _metrics_ refer to the statistics being monitored. Within Prometheus are different forms of metric. The exposure of these metrics from a kdb+ session allows for the monitoring a kdb+ process with Prometheus.
37+
38+
There are [four types of metric](https://prometheus.io/docs/concepts/metric_types/) in Prometheus:
39+
40+
```txt
41+
counter
42+
gauge
43+
histogram
44+
summary
45+
```
46+
47+
These are classified as either _Single-value_ or _Sample_ metrics
48+
49+
- Single-value metrics
50+
51+
Both `counter` and `gauge` are single-value metrics, providing a number per instance.
52+
53+
When updating a single-value metric, a single number will be modified. On a request, this number will be reported directly as the metric value.
54+
55+
- Sample metrics
56+
57+
Both `histogram` and `summary` are aggregate metrics, providing summary statistics (defined by the metric params) per instance.
58+
59+
When updating a sample metric, a list of numeric values will be appended to. On request, this list will be used to construct the metric values, depending on the metric type and params.
60+
61+
## Status
62+
63+
The interface is currently available under an Apache 2.0 licence and is supported on a best-efforts basis by the Fusion team. The interface is currently in active development, with additional functionality to be released on an ongoing basis.
64+
65+
66+
[Issues and feature requests](../../../issues)
67+
68+
[Guide to contributing](../CONTRIBUTING.md)
69+

‎docs/event-handlers.md

+238
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,238 @@
1+
# Prometheus metric event handlers
2+
3+
4+
Metric updates are generally carried out inside event handlers. By overwriting these handlers, users can update metric values in response to various triggers ranging from HTTP requests to timer events.
5+
6+
The functions outlined below can be modified to allow a user to monitor events outside those exposed in `exporter.q`
7+
8+
`.prom.` **event handlers**<br>
9+
[`on_poll`](#promon_poll) Prometheus poll request<br>
10+
[`on_pc`](#promon_pc) IPC socket connection closing<br>
11+
[`on_po`](#promon_po) IPC socket connection opening<br>
12+
[`on_wc`](#promon_wc) Websocket connection closing<br>
13+
[`on_wo`](#promon_wo) Websocket connection opening<br>
14+
[`after_pg`](#promafer_pg) Synchronous IPC socket request handler, call after execution<br>
15+
[`before_pg`](#prombefore_pg) Synchronous IPC socket request handler, call before execution<br>
16+
[`after_ps`](#promafter_ps) Asynchronous IPC socket request handler, call after execution<br>
17+
[`before_ps`](#prombefore_ps) Asynchronous IPC socket request handler, call before execution<br>
18+
[`after_ph`](#promafter_ph) HTTP GET request handler, call after execution<br>
19+
[`before_ph`](#prombefore_ph) HTTP GET request handler, call before execution<br>
20+
[`after_pp`](#promafter_pp) HTTP POST request handler, call after execution<br>
21+
[`before_pp`](#prombefore_pp) HTTP POST request handler, call before execution<br>
22+
[`after_ws`](#promafter_ws) Websocket request handler, call after execution<br>
23+
[`before_ws`](#prombefore_ws) Websocket request handler, call before execution<br>
24+
[`after_ts`](#promafter_ts) Timer event handler, call after execution<br>
25+
[`before_ts`](#prombefore_ts) Timer event handler, call after execution
26+
27+
:point_right:
28+
[Example invocations of these event handlers](../examples/kdb_user_example.q)
29+
30+
Once the relevant event handlers have been defined to update the metric values, initialize the library with a call to [`.prom.init`](reference.md#initialize-library)
31+
32+
:warning:
33+
Updating `.z.*` handlers after the call to `.prom.init` will overwrite the Prometheus logic.
34+
Correct usage is to load all process logic before loading the Prometheus library.
35+
36+
37+
## `.prom.after_pg`
38+
39+
_Synchronous IPC socket request handler, called after execution_
40+
41+
```txt
42+
.prom.after_pg[tmp;msg;res]
43+
```
44+
45+
46+
Where
47+
48+
- `tmp` is the object returned by `.prom.before_pg`
49+
- `msg` is the object that was executed
50+
- `res` is the object returned by the execution of `msg`
51+
52+
53+
## `.prom.after_ph`
54+
55+
_HTTP GET request handler, called after execution_
56+
57+
```txt
58+
.prom.after_ph[tmp;(requestText;requestHeaderAsDictionary);res]
59+
```
60+
61+
62+
Where
63+
64+
- `tmp` is the object returned by `.prom.before_ph`
65+
- `(requestText;requestHeaderAsDictionary)` is a HTTP GET request
66+
- `res` is the object returned by the execution of `msg`
67+
68+
69+
## `.prom.after_pp`
70+
71+
_HTTP POST request handler, called after execution_
72+
73+
```txt
74+
.prom.after_pp[tmp;(requestText;requestHeaderAsDictionary);res]
75+
```
76+
77+
78+
Where
79+
80+
- `tmp` is the object returned by `.prom.before_pp`
81+
- `(requestText;requestHeaderAsDictionary)` is a HTTP POST request
82+
- `res` is the object returned by the execution of `msg`
83+
84+
85+
## `.prom.after_ps`
86+
87+
_Asynchronous IPC socket request handler, called after execution_
88+
89+
```txt
90+
.prom.after_ps[tmp;msg;res]
91+
```
92+
93+
94+
Where
95+
96+
- `tmp` is the object returned by `.prom.before_ps`
97+
- `msg` is the object that was executed
98+
- `res` is the object returned by the execution of `msg`
99+
100+
101+
## `.prom.after_ts`
102+
103+
_Timer event handler, called after execution_
104+
105+
```txt
106+
.prom.after_ts[tmp;dtm;res]
107+
```
108+
109+
110+
Where
111+
112+
- `tmp` is the object returned by `.prom.before_ts`
113+
- `dtm` is the timestamp at the start of execution
114+
- `res` is the object returned by the execution of `dtm`
115+
116+
117+
## `.prom.after_ws`
118+
119+
_Websocket request handler, called after execution_
120+
121+
```txt
122+
.prom.after_ws[tmp;msg;res]
123+
```
124+
125+
126+
Where
127+
128+
- `tmp` is the object returned by `.prom.before_ws`
129+
- `msg` is the object that was executed
130+
- `res` is the object returned by the execution of `msg`
131+
132+
133+
## `.prom.before_pg`
134+
135+
_Synchronous IPC socket request handler, called before execution_
136+
137+
```txt
138+
.prom.before_pg msg
139+
```
140+
141+
142+
Where `msg` is the object to be executed, returns a `tmp` object to be passed to the _after_ handler.
143+
144+
145+
## `.prom.before_ph`
146+
147+
_HTTP GET request handler, called before execution_
148+
149+
```txt
150+
.prom.before_ph(requestText;requestHeaderAsDictionary)
151+
```
152+
153+
154+
Where `(requestText;requestHeaderAsDictionary)` is an HTTP GET request, returns a `tmp` object to be passed to the _after_ handler.
155+
156+
157+
## `.prom.before_pp`
158+
159+
_HTTP POST request handler, called before execution_
160+
161+
```txt
162+
.prom.before_pp(requestText;requestHeaderAsDictionary)
163+
```
164+
165+
166+
Where `(requestText;requestHeaderAsDictionary)` is an HTTP POST request, returns a `tmp` object to be passed to the _after_ handler.
167+
168+
169+
## `.prom.before_ps`
170+
171+
_Asynchronous IPC socket request handler, called before execution_
172+
173+
```txt
174+
.prom.before_ps msg
175+
```
176+
177+
178+
Where `msg` is the object to be executed, returns a `tmp` object to be passed to the _after_ handler.
179+
180+
181+
## `.prom.before_ts`
182+
183+
_Timer event handler, called before execution_
184+
185+
```txt
186+
.prom.before_ts dtm
187+
```
188+
189+
190+
Where `dtm` is the timestamp at the start of execution, returns a `tmp` object to be passed to the _after_ handler.
191+
192+
193+
## `.prom.before_ws`
194+
195+
_Websocket request handler, called before execution_
196+
197+
```txt
198+
.prom.before_ws msg
199+
```
200+
201+
202+
Where `msg` is the object to be executed, returns a `tmp` object to be passed to the _after_ handler.
203+
204+
205+
## `.prom.on_pc`
206+
207+
_Socket close handler_
208+
209+
```txt
210+
.prom.on_pc hdl
211+
```
212+
213+
214+
Where `hdl` is the handle of a socket connection, closes the socket.
215+
216+
217+
## `.prom.on_po`
218+
219+
_Socket open handler_
220+
221+
```txt
222+
.prom.on_po hdl
223+
```
224+
225+
226+
Where `hdl` is the handle of a socket connection, opens the socket.
227+
228+
229+
## `.prom.on_poll`
230+
231+
_Prometheus poll request handler_
232+
233+
```txt
234+
.prom.on_poll(requestText;requestHeaderAsDictionary)
235+
```
236+
237+
Where `(requestText;requestHeaderAsDictionary)` is an HTTP request
238+

0 commit comments

Comments
 (0)