Skip to content

Commit e97caed

Browse files
authoredMar 19, 2025··
Merge pull request #2491 from pyth-network/price-pusher-metrics
feat(apps/price_pusher): add prom metrics
2 parents d61dca2 + 19ebf18 commit e97caed

18 files changed

+2207
-321
lines changed
 

‎apps/price_pusher/README.md

+114
Original file line numberDiff line numberDiff line change
@@ -259,3 +259,117 @@ pushed twice and you won't pay additional costs most of the time.** However, the
259259
conditions in the RPCs because they are often behind a load balancer which can sometimes cause rejected
260260
transactions to land on-chain. You can reduce the chances of additional cost overhead by reducing the
261261
pushing frequency.
262+
263+
## Prometheus Metrics
264+
265+
The price_pusher now supports Prometheus metrics to monitor the health and performance of the price update service. Metrics are exposed via an HTTP endpoint that can be scraped by Prometheus.
266+
267+
### Available Metrics
268+
269+
The following metrics are available:
270+
271+
- **pyth_price_last_published_time** (Gauge): The last published time of a price feed in unix timestamp, labeled by price_id and alias
272+
- **pyth_price_update_attempts_total** (Counter): Total number of price update attempts with their trigger condition and status, labeled by price_id, alias, trigger, and status
273+
- **pyth_price_feeds_total** (Gauge): Total number of price feeds being monitored
274+
- **pyth_wallet_balance** (Gauge): Current wallet balance of the price pusher in native token units, labeled by wallet_address and network
275+
276+
### Configuration
277+
278+
Metrics are enabled by default and can be configured using the following command-line options:
279+
280+
- `--enable-metrics`: Enable or disable the Prometheus metrics server (default: true)
281+
- `--metrics-port`: Port for the Prometheus metrics server (default: 9090)
282+
283+
Example:
284+
285+
```bash
286+
pnpm run dev evm --config config.evm.mainnet.json --metrics-port 9091
287+
```
288+
289+
### Running Locally with Docker
290+
291+
You can run the monitoring stack (Prometheus and Grafana) using the provided docker-compose configuration:
292+
293+
1. Use the sample docker-compose file for metrics:
294+
295+
```bash
296+
docker-compose -f docker-compose.metrics.sample.yaml up
297+
```
298+
299+
This will start:
300+
- Prometheus server on port 9090 with the alerts configured in alerts.sample.yml
301+
- Grafana server on port 3000 with default credentials (admin/admin)
302+
303+
The docker-compose.metrics.sample.yaml file includes a pre-configured Grafana dashboard (see the [Dashboard](#dashboard) section below) that displays all the metrics mentioned above. This dashboard provides monitoring of your price pusher operations with panels for configured feeds, active feeds, wallet balance, update statistics, and error tracking. The dashboard is automatically provisioned when you start the stack with docker-compose.
304+
305+
### Example Grafana Queries
306+
307+
Here are some example Grafana queries to monitor your price feeds:
308+
309+
1. Last published time for each price feed:
310+
311+
```
312+
pyth_price_last_published_time
313+
```
314+
315+
2. Number of price updates in the last hour:
316+
317+
```
318+
sum(increase(pyth_price_update_attempts_total{status="success"}[1h]))
319+
```
320+
321+
3. Price feeds not updated in the last hour:
322+
323+
```
324+
time() - pyth_price_last_published_time > 3600
325+
```
326+
327+
4. Distribution of update conditions:
328+
329+
```
330+
sum by (condition) (increase(pyth_update_conditions_total[$__range]))
331+
```
332+
333+
5. Monitor wallet balances:
334+
335+
```
336+
pyth_wallet_balance
337+
```
338+
339+
6. Detect low wallet balances (below 0.1 tokens):
340+
341+
```
342+
pyth_wallet_balance < 0.1
343+
```
344+
345+
### Dashboard
346+
347+
The docker-compose setup includes a pre-configured Grafana dashboard (`grafana-dashboard.sample.json`) that provides monitoring of your price pusher operations. The dashboard includes the following panels:
348+
349+
- **Configured Price Feeds**: Shows the number of price feeds configured in your price-config file.
350+
- **Active Price Feeds**: Displays the number of price feeds currently being actively monitored.
351+
- **Time Since Last Update**: Shows how long it's been since the last successful price update was published on-chain.
352+
- **Price Feeds List**: A table listing all configured price feeds with their details.
353+
- **Successful Updates (Current Range)**: Graph showing the number of successful price updates over the current range with timeline.
354+
- **Update Conditions Distribution**: Pie chart showing the distribution of update conditions (YES/NO/EARLY) over the selected time range.
355+
- **Wallet Balance**: Current balance of your wallet in native token units.
356+
- **Wallet Balance Over Time**: Graph tracking your wallet balance over time to monitor consumption.
357+
- **Failed Updates (Current Range)**: Graph showing the number of failed price updates over the current range with timeline.
358+
359+
When you first start the monitoring stack, the dashboard may show "No data" in the panels until the price pusher has been running for some time and has collected sufficient metrics.
360+
361+
This dashboard is automatically provisioned when you start the docker-compose stack and provides visibility into the health and performance of your price pusher deployment.
362+
363+
### Alerting
364+
365+
The price pusher includes pre-configured Prometheus alerting rules in the `alerts.sample.yml` file. These rules monitor various aspects of the price pusher's operation, including:
366+
367+
- Price feeds not being updated for an extended period (>1 hour)
368+
- High error rates in price update attempts
369+
- No successful price updates across all feeds in the last 30 minutes
370+
- Service availability monitoring
371+
- Low wallet balances with two severity levels:
372+
- Warning: Balance below 0.1 native tokens
373+
- Critical: Balance below 0.01 native tokens (transactions may fail soon)
374+
375+
When using the docker-compose setup, these alerts are automatically loaded into Prometheus and can be viewed in the Alerting section of Grafana after setting up the Prometheus data source.

‎apps/price_pusher/alerts.sample.yml

+56
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
groups:
2+
- name: price_pusher_alerts
3+
rules:
4+
- alert: PriceFeedNotUpdated
5+
expr: time() - pyth_price_last_published_time > 3600
6+
for: 5m
7+
labels:
8+
severity: warning
9+
annotations:
10+
summary: "Price feed not updated"
11+
description: "Price feed {{ $labels.alias }} has not been updated for more than 1 hour"
12+
13+
- alert: HighErrorRate
14+
expr: sum(increase(pyth_price_update_attempts_total{status="error"}[15m])) > 5
15+
for: 5m
16+
labels:
17+
severity: warning
18+
annotations:
19+
summary: "High error rate in price updates"
20+
description: "There have been more than 5 errors in the last 15 minutes"
21+
22+
- alert: NoRecentPriceUpdates
23+
expr: sum(increase(pyth_price_update_attempts_total{status="success"}[30m])) == 0
24+
for: 5m
25+
labels:
26+
severity: critical
27+
annotations:
28+
summary: "No recent price updates"
29+
description: "No price updates have been pushed in the last 30 minutes"
30+
31+
- alert: PricePusherDown
32+
expr: up{job=~"price_pusher.*"} == 0
33+
for: 1m
34+
labels:
35+
severity: critical
36+
annotations:
37+
summary: "Price pusher service is down"
38+
description: "The price pusher service {{ $labels.instance }} is down"
39+
40+
- alert: WalletBalanceLow
41+
expr: pyth_wallet_balance < 0.1
42+
for: 5m
43+
labels:
44+
severity: warning
45+
annotations:
46+
summary: "Wallet balance is getting low"
47+
description: "Wallet {{ $labels.wallet_address }} on network {{ $labels.network }} has balance below 0.1 native tokens"
48+
49+
- alert: WalletBalanceCritical
50+
expr: pyth_wallet_balance < 0.01
51+
for: 2m
52+
labels:
53+
severity: critical
54+
annotations:
55+
summary: "Wallet balance critically low"
56+
description: "Wallet {{ $labels.wallet_address }} on network {{ $labels.network }} has balance below 0.01 native tokens. Transactions may fail soon!"
+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
apiVersion: 1
2+
3+
providers:
4+
- name: 'Pyth Price Pusher'
5+
orgId: 1
6+
folder: ''
7+
type: file
8+
disableDeletion: false
9+
editable: true
10+
options:
11+
path: /var/lib/grafana/dashboards
+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
apiVersion: 1
2+
3+
datasources:
4+
- name: Prometheus
5+
type: prometheus
6+
access: proxy
7+
url: http://prometheus:9090
8+
isDefault: true
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
version: "3"
2+
3+
services:
4+
prometheus:
5+
image: prom/prometheus:latest
6+
container_name: prometheus
7+
ports:
8+
- "9090:9090"
9+
volumes:
10+
- ./prometheus.sample.yml:/etc/prometheus/prometheus.yml
11+
- ./alerts.sample.yml:/etc/prometheus/alerts.yml
12+
command:
13+
- "--config.file=/etc/prometheus/prometheus.yml"
14+
- "--storage.tsdb.path=/prometheus"
15+
- "--web.console.libraries=/usr/share/prometheus/console_libraries"
16+
- "--web.console.templates=/usr/share/prometheus/consoles"
17+
networks:
18+
- monitoring
19+
20+
grafana:
21+
image: grafana/grafana:latest
22+
container_name: grafana
23+
ports:
24+
- "3000:3000"
25+
volumes:
26+
- grafana-storage:/var/lib/grafana
27+
- ./grafana-dashboard.sample.json:/var/lib/grafana/dashboards/pyth-price-pusher-dashboard.json
28+
- ./dashboard.sample.yml:/etc/grafana/provisioning/dashboards/dashboard.yml
29+
- ./datasource.sample.yml:/etc/grafana/provisioning/datasources/datasource.yml
30+
environment:
31+
- GF_SECURITY_ADMIN_USER=admin
32+
- GF_SECURITY_ADMIN_PASSWORD=admin
33+
- GF_USERS_ALLOW_SIGN_UP=false
34+
depends_on:
35+
- prometheus
36+
networks:
37+
- monitoring
38+
39+
networks:
40+
monitoring:
41+
driver: bridge
42+
43+
volumes:
44+
grafana-storage:

‎apps/price_pusher/grafana-dashboard.sample.json

+955
Large diffs are not rendered by default.

‎apps/price_pusher/package.json

+4-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "@pythnetwork/price-pusher",
3-
"version": "9.1.1",
3+
"version": "9.1.2",
44
"description": "Pyth Price Pusher",
55
"homepage": "https://pyth.network",
66
"main": "lib/index.js",
@@ -45,6 +45,7 @@
4545
"license": "Apache-2.0",
4646
"devDependencies": {
4747
"@types/ethereum-protocol": "^1.0.2",
48+
"@types/express": "^4.17.21",
4849
"@types/jest": "^27.4.1",
4950
"@types/yargs": "^17.0.10",
5051
"@typescript-eslint/eslint-plugin": "^6.0.0",
@@ -76,11 +77,13 @@
7677
"@ton/ton": "^15.1.0",
7778
"@types/pino": "^7.0.5",
7879
"aptos": "^1.8.5",
80+
"express": "^4.18.2",
7981
"fuels": "^0.94.5",
8082
"jito-ts": "^3.0.1",
8183
"joi": "^17.6.0",
8284
"near-api-js": "^3.0.2",
8385
"pino": "^9.2.0",
86+
"prom-client": "^15.1.0",
8487
"viem": "^2.19.4",
8588
"yaml": "^2.1.1",
8689
"yargs": "^17.5.1"

‎apps/price_pusher/price-config.stable.sample.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
price_deviation: 0.5
1414
confidence_ratio: 0.1
1515
- alias: PYTH/USD
16-
id: 2f95862b045670cd22bee3114c39763a4a08beeb663b145d283c31d7d1101c23
16+
id: 0bbf28e9a841a1cc788f6a361b17ca072d0ea3098a1e5df1c3922d06719579ff
1717
time_difference: 60
1818
price_deviation: 0.5
1919
confidence_ratio: 1
+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
global:
2+
scrape_interval: 15s
3+
evaluation_interval: 15s
4+
5+
scrape_configs:
6+
- job_name: "price_pusher"
7+
static_configs:
8+
- targets: ["host.docker.internal:9091"]
9+
relabel_configs:
10+
- source_labels: [__address__]
11+
target_label: instance
12+
replacement: "price_pusher"
13+
14+
alerting:
15+
alertmanagers:
16+
- static_configs:
17+
- targets:
18+
# - alertmanager:9093
19+
20+
# Alert rules
21+
rule_files:
22+
- "alerts.sample.yml"
+38
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
import { PricePusherMetrics } from "./metrics";
2+
import { Logger } from "pino";
3+
import { DurationInSeconds } from "./utils";
4+
import { IBalanceTracker } from "./interface";
5+
import { EvmBalanceTracker } from "./evm/balance-tracker";
6+
import { SuperWalletClient } from "./evm/super-wallet";
7+
8+
/**
9+
* Parameters for creating an EVM balance tracker
10+
*/
11+
export interface CreateEvmBalanceTrackerParams {
12+
client: SuperWalletClient;
13+
address: `0x${string}`;
14+
network: string;
15+
updateInterval: DurationInSeconds;
16+
metrics: PricePusherMetrics;
17+
logger: Logger;
18+
}
19+
20+
/**
21+
* Factory function to create a balance tracker for EVM chains
22+
*/
23+
export function createEvmBalanceTracker(
24+
params: CreateEvmBalanceTrackerParams,
25+
): IBalanceTracker {
26+
return new EvmBalanceTracker({
27+
client: params.client,
28+
address: params.address,
29+
network: params.network,
30+
updateInterval: params.updateInterval,
31+
metrics: params.metrics,
32+
logger: params.logger,
33+
});
34+
}
35+
36+
// Additional factory functions for other chains would follow the same pattern:
37+
// export function createSuiBalanceTracker(params: CreateSuiBalanceTrackerParams): IBalanceTracker { ... }
38+
// export function createSolanaBalanceTracker(params: CreateSolanaBalanceTrackerParams): IBalanceTracker { ... }

‎apps/price_pusher/src/controller.ts

+78-1
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,12 @@ import { DurationInSeconds, sleep } from "./utils";
33
import { IPriceListener, IPricePusher } from "./interface";
44
import { PriceConfig, shouldUpdate, UpdateCondition } from "./price-config";
55
import { Logger } from "pino";
6+
import { PricePusherMetrics } from "./metrics";
67

78
export class Controller {
89
private pushingFrequency: DurationInSeconds;
10+
private metrics?: PricePusherMetrics;
11+
912
constructor(
1013
private priceConfigs: PriceConfig[],
1114
private sourcePriceListener: IPriceListener,
@@ -14,9 +17,14 @@ export class Controller {
1417
private logger: Logger,
1518
config: {
1619
pushingFrequency: DurationInSeconds;
20+
metrics?: PricePusherMetrics;
1721
},
1822
) {
1923
this.pushingFrequency = config.pushingFrequency;
24+
this.metrics = config.metrics;
25+
26+
// Set the number of price feeds if metrics are enabled
27+
this.metrics?.setPriceFeedsTotal(this.priceConfigs.length);
2028
}
2129

2230
async start() {
@@ -38,18 +46,34 @@ export class Controller {
3846

3947
for (const priceConfig of this.priceConfigs) {
4048
const priceId = priceConfig.id;
49+
const alias = priceConfig.alias;
4150

4251
const targetLatestPrice =
4352
this.targetPriceListener.getLatestPriceInfo(priceId);
4453
const sourceLatestPrice =
4554
this.sourcePriceListener.getLatestPriceInfo(priceId);
4655

56+
// Update metrics for the last published time if available
57+
if (this.metrics && targetLatestPrice) {
58+
this.metrics.updateLastPublishedTime(
59+
priceId,
60+
alias,
61+
targetLatestPrice,
62+
);
63+
}
64+
4765
const priceShouldUpdate = shouldUpdate(
4866
priceConfig,
4967
sourceLatestPrice,
5068
targetLatestPrice,
5169
this.logger,
5270
);
71+
72+
// Record update condition in metrics
73+
if (this.metrics) {
74+
this.metrics.recordUpdateCondition(priceId, alias, priceShouldUpdate);
75+
}
76+
5377
if (priceShouldUpdate == UpdateCondition.YES) {
5478
pushThresholdMet = true;
5579
}
@@ -75,7 +99,60 @@ export class Controller {
7599

76100
// note that the priceIds are without leading "0x"
77101
const priceIds = pricesToPush.map((priceConfig) => priceConfig.id);
78-
this.targetChainPricePusher.updatePriceFeed(priceIds, pubTimesToPush);
102+
103+
try {
104+
await this.targetChainPricePusher.updatePriceFeed(
105+
priceIds,
106+
pubTimesToPush,
107+
);
108+
109+
// Record successful updates
110+
if (this.metrics) {
111+
for (const config of pricesToPush) {
112+
const triggerValue =
113+
shouldUpdate(
114+
config,
115+
this.sourcePriceListener.getLatestPriceInfo(config.id),
116+
this.targetPriceListener.getLatestPriceInfo(config.id),
117+
this.logger,
118+
) === UpdateCondition.YES
119+
? "yes"
120+
: "early";
121+
122+
this.metrics.recordPriceUpdate(
123+
config.id,
124+
config.alias,
125+
triggerValue,
126+
);
127+
}
128+
}
129+
} catch (error) {
130+
this.logger.error(
131+
{ error, priceIds },
132+
"Error pushing price updates to chain",
133+
);
134+
135+
// Record errors in metrics
136+
if (this.metrics) {
137+
for (const config of pricesToPush) {
138+
const triggerValue =
139+
shouldUpdate(
140+
config,
141+
this.sourcePriceListener.getLatestPriceInfo(config.id),
142+
this.targetPriceListener.getLatestPriceInfo(config.id),
143+
this.logger,
144+
) === UpdateCondition.YES
145+
? "yes"
146+
: "early";
147+
148+
this.metrics.recordPriceUpdateError(
149+
config.id,
150+
config.alias,
151+
triggerValue,
152+
);
153+
}
154+
}
155+
}
79156
} else {
80157
this.logger.info("None of the checks were triggered. No push needed.");
81158
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
import { SuperWalletClient } from "./super-wallet";
2+
import { BaseBalanceTracker, BaseBalanceTrackerConfig } from "../interface";
3+
4+
/**
5+
* EVM-specific configuration for balance tracker
6+
*/
7+
export interface EvmBalanceTrackerConfig extends BaseBalanceTrackerConfig {
8+
/** EVM wallet client */
9+
client: SuperWalletClient;
10+
/** EVM address with 0x prefix */
11+
address: `0x${string}`;
12+
}
13+
14+
/**
15+
* EVM-specific implementation of the balance tracker
16+
*/
17+
export class EvmBalanceTracker extends BaseBalanceTracker {
18+
private client: SuperWalletClient;
19+
private evmAddress: `0x${string}`;
20+
21+
constructor(config: EvmBalanceTrackerConfig) {
22+
super({
23+
...config,
24+
logger: config.logger.child({ module: "EvmBalanceTracker" }),
25+
});
26+
27+
this.client = config.client;
28+
this.evmAddress = config.address;
29+
}
30+
31+
/**
32+
* EVM-specific implementation of balance update
33+
*/
34+
protected async updateBalance(): Promise<void> {
35+
try {
36+
const balance = await this.client.getBalance({
37+
address: this.evmAddress,
38+
});
39+
40+
this.metrics.updateWalletBalance(this.address, this.network, balance);
41+
this.logger.debug(
42+
`Updated EVM wallet balance: ${this.address} = ${balance.toString()}`,
43+
);
44+
} catch (error) {
45+
this.logger.error(
46+
{ error },
47+
"Error fetching EVM wallet balance for metrics",
48+
);
49+
}
50+
}
51+
}

‎apps/price_pusher/src/evm/command.ts

+34-2
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,8 @@ import pino from "pino";
1111
import { createClient } from "./super-wallet";
1212
import { createPythContract } from "./pyth-contract";
1313
import { isWsEndpoint, filterInvalidPriceItems } from "../utils";
14+
import { PricePusherMetrics } from "../metrics";
15+
import { createEvmBalanceTracker } from "../balance-tracker";
1416

1517
export default {
1618
command: "evm",
@@ -83,6 +85,8 @@ export default {
8385
...options.pushingFrequency,
8486
...options.logLevel,
8587
...options.controllerLogLevel,
88+
...options.enableMetrics,
89+
...options.metricsPort,
8690
},
8791
handler: async function (argv: any) {
8892
// FIXME: type checks for this
@@ -103,6 +107,8 @@ export default {
103107
updateFeeMultiplier,
104108
logLevel,
105109
controllerLogLevel,
110+
enableMetrics,
111+
metricsPort,
106112
} = argv;
107113
console.log("***** priceServiceEndpoint *****", priceServiceEndpoint);
108114

@@ -131,6 +137,14 @@ export default {
131137

132138
priceItems = existingPriceItems;
133139

140+
// Initialize metrics if enabled
141+
let metrics: PricePusherMetrics | undefined;
142+
if (enableMetrics) {
143+
metrics = new PricePusherMetrics(logger.child({ module: "Metrics" }));
144+
metrics.start(metricsPort);
145+
logger.info(`Metrics server started on port ${metricsPort}`);
146+
}
147+
134148
const pythListener = new PythPriceListener(
135149
hermesClient,
136150
priceItems,
@@ -183,9 +197,27 @@ export default {
183197
evmListener,
184198
evmPusher,
185199
logger.child({ module: "Controller" }, { level: controllerLogLevel }),
186-
{ pushingFrequency },
200+
{
201+
pushingFrequency,
202+
metrics,
203+
},
187204
);
188205

189-
controller.start();
206+
// Create and start the balance tracker if metrics are enabled
207+
if (metrics) {
208+
const balanceTracker = createEvmBalanceTracker({
209+
client,
210+
address: client.account.address,
211+
network: await client.getChainId().then((id) => id.toString()),
212+
updateInterval: pushingFrequency,
213+
metrics,
214+
logger,
215+
});
216+
217+
// Start the balance tracker
218+
await balanceTracker.start();
219+
}
220+
221+
await controller.start();
190222
},
191223
};

‎apps/price_pusher/src/index.ts

+3
Original file line numberDiff line numberDiff line change
@@ -9,13 +9,16 @@ import near from "./near/command";
99
import solana from "./solana/command";
1010
import fuel from "./fuel/command";
1111
import ton from "./ton/command";
12+
import { enableMetrics, metricsPort } from "./options";
1213

1314
yargs(hideBin(process.argv))
1415
.parserConfiguration({
1516
"parse-numbers": false,
1617
})
1718
.config("config")
1819
.global("config")
20+
.option("enable-metrics", enableMetrics["enable-metrics"])
21+
.option("metrics-port", metricsPort["metrics-port"])
1922
.command(evm)
2023
.command(fuel)
2124
.command(injective)

‎apps/price_pusher/src/interface.ts

+96
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
import { HexString, UnixTimestamp } from "@pythnetwork/hermes-client";
22
import { DurationInSeconds } from "./utils";
3+
import { Logger } from "pino";
4+
import { PricePusherMetrics } from "./metrics";
35

46
export type PriceItem = {
57
id: HexString;
@@ -82,3 +84,97 @@ export interface IPricePusher {
8284
pubTimesToPush: UnixTimestamp[],
8385
): Promise<void>;
8486
}
87+
88+
/**
89+
* Common configuration properties for all balance trackers
90+
*/
91+
export interface BaseBalanceTrackerConfig {
92+
/** Address of the wallet to track */
93+
address: string;
94+
/** Name/ID of the network/chain */
95+
network: string;
96+
/** How often to update the balance */
97+
updateInterval: DurationInSeconds;
98+
/** Metrics instance to report balance updates */
99+
metrics: PricePusherMetrics;
100+
/** Logger instance */
101+
logger: Logger;
102+
}
103+
104+
/**
105+
* Interface for all balance trackers to implement
106+
* Each chain will have its own implementation of this interface
107+
*/
108+
export interface IBalanceTracker {
109+
/**
110+
* Start tracking the wallet balance
111+
*/
112+
start(): Promise<void>;
113+
114+
/**
115+
* Stop tracking the wallet balance
116+
*/
117+
stop(): void;
118+
}
119+
120+
/**
121+
* Abstract base class that implements common functionality for all balance trackers
122+
*/
123+
export abstract class BaseBalanceTracker implements IBalanceTracker {
124+
protected address: string;
125+
protected network: string;
126+
protected updateInterval: DurationInSeconds;
127+
protected metrics: PricePusherMetrics;
128+
protected logger: Logger;
129+
protected isRunning: boolean = false;
130+
131+
constructor(config: BaseBalanceTrackerConfig) {
132+
this.address = config.address;
133+
this.network = config.network;
134+
this.updateInterval = config.updateInterval;
135+
this.metrics = config.metrics;
136+
this.logger = config.logger;
137+
}
138+
139+
public async start(): Promise<void> {
140+
if (this.isRunning) {
141+
return;
142+
}
143+
144+
this.isRunning = true;
145+
146+
// Initial balance update
147+
await this.updateBalance();
148+
149+
// Start the update loop
150+
this.startUpdateLoop();
151+
}
152+
153+
private async startUpdateLoop(): Promise<void> {
154+
// We're using dynamic import to avoid circular dependencies
155+
const { sleep } = await import("./utils");
156+
157+
// Run in a loop to regularly update the balance
158+
for (;;) {
159+
// Wait first, since we already did the initial update in start()
160+
await sleep(this.updateInterval * 1000);
161+
162+
// Only continue if we're still running
163+
if (!this.isRunning) {
164+
break;
165+
}
166+
167+
await this.updateBalance();
168+
}
169+
}
170+
171+
/**
172+
* Chain-specific balance update implementation
173+
* Each chain will implement this method differently
174+
*/
175+
protected abstract updateBalance(): Promise<void>;
176+
177+
public stop(): void {
178+
this.isRunning = false;
179+
}
180+
}

‎apps/price_pusher/src/metrics.ts

+153
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
import { Registry, Counter, Gauge } from "prom-client";
2+
import express from "express";
3+
import { PriceInfo } from "./interface";
4+
import { Logger } from "pino";
5+
import { UpdateCondition } from "./price-config";
6+
7+
// Define the metrics we want to track
8+
export class PricePusherMetrics {
9+
private registry: Registry;
10+
private server: express.Express;
11+
private logger: Logger;
12+
13+
// Metrics for price feed updates
14+
public lastPublishedTime: Gauge<string>;
15+
public priceUpdateAttempts: Counter<string>;
16+
public priceFeedsTotal: Gauge<string>;
17+
// Wallet metrics
18+
public walletBalance: Gauge<string>;
19+
20+
constructor(logger: Logger) {
21+
this.logger = logger;
22+
this.registry = new Registry();
23+
this.server = express();
24+
25+
// Register the default metrics (memory, CPU, etc.)
26+
this.registry.setDefaultLabels({ app: "price_pusher" });
27+
28+
// Create metrics
29+
this.lastPublishedTime = new Gauge({
30+
name: "pyth_price_last_published_time",
31+
help: "The last published time of a price feed in unix timestamp",
32+
labelNames: ["price_id", "alias"],
33+
registers: [this.registry],
34+
});
35+
36+
this.priceUpdateAttempts = new Counter({
37+
name: "pyth_price_update_attempts_total",
38+
help: "Total number of price update attempts with their trigger condition and status",
39+
labelNames: ["price_id", "alias", "trigger", "status"],
40+
registers: [this.registry],
41+
});
42+
43+
this.priceFeedsTotal = new Gauge({
44+
name: "pyth_price_feeds_total",
45+
help: "Total number of price feeds being monitored",
46+
registers: [this.registry],
47+
});
48+
49+
// Wallet balance metric
50+
this.walletBalance = new Gauge({
51+
name: "pyth_wallet_balance",
52+
help: "Current wallet balance of the price pusher in native token units",
53+
labelNames: ["wallet_address", "network"],
54+
registers: [this.registry],
55+
});
56+
57+
// Setup the metrics endpoint
58+
this.server.get("/metrics", async (req, res) => {
59+
res.set("Content-Type", this.registry.contentType);
60+
res.end(await this.registry.metrics());
61+
});
62+
}
63+
64+
// Start the metrics server
65+
public start(port: number): void {
66+
this.server.listen(port, () => {
67+
this.logger.info(`Metrics server started on port ${port}`);
68+
});
69+
}
70+
71+
// Update the last published time for a price feed
72+
public updateLastPublishedTime(
73+
priceId: string,
74+
alias: string,
75+
priceInfo: PriceInfo,
76+
): void {
77+
this.lastPublishedTime.set(
78+
{ price_id: priceId, alias },
79+
priceInfo.publishTime,
80+
);
81+
}
82+
83+
// Record a successful price update
84+
public recordPriceUpdate(
85+
priceId: string,
86+
alias: string,
87+
trigger: string = "yes",
88+
): void {
89+
this.priceUpdateAttempts.inc({
90+
price_id: priceId,
91+
alias,
92+
trigger: trigger.toLowerCase(),
93+
status: "success",
94+
});
95+
}
96+
97+
// Record update condition status (YES/NO/EARLY)
98+
public recordUpdateCondition(
99+
priceId: string,
100+
alias: string,
101+
condition: UpdateCondition,
102+
): void {
103+
const triggerLabel = UpdateCondition[condition].toLowerCase();
104+
// Only record as 'skipped' when the condition is NO
105+
if (condition === UpdateCondition.NO) {
106+
this.priceUpdateAttempts.inc({
107+
price_id: priceId,
108+
alias,
109+
trigger: triggerLabel,
110+
status: "skipped",
111+
});
112+
}
113+
// YES and EARLY don't increment the counter here - they'll be counted
114+
// when recordPriceUpdate or recordPriceUpdateError is called
115+
}
116+
117+
// Record a price update error
118+
public recordPriceUpdateError(
119+
priceId: string,
120+
alias: string,
121+
trigger: string = "yes",
122+
): void {
123+
this.priceUpdateAttempts.inc({
124+
price_id: priceId,
125+
alias,
126+
trigger: trigger.toLowerCase(),
127+
status: "error",
128+
});
129+
}
130+
131+
// Set the number of price feeds
132+
public setPriceFeedsTotal(count: number): void {
133+
this.priceFeedsTotal.set(count);
134+
}
135+
136+
// Update wallet balance
137+
public updateWalletBalance(
138+
walletAddress: string,
139+
network: string,
140+
balance: bigint | number,
141+
): void {
142+
// Convert to number for compatibility with prometheus
143+
const balanceNum =
144+
typeof balance === "bigint" ? Number(balance) / 1e18 : balance;
145+
this.walletBalance.set(
146+
{ wallet_address: walletAddress, network },
147+
balanceNum,
148+
);
149+
this.logger.debug(
150+
`Updated wallet balance metric: ${walletAddress} = ${balanceNum}`,
151+
);
152+
}
153+
}

‎apps/price_pusher/src/options.ts

+18
Original file line numberDiff line numberDiff line change
@@ -76,3 +76,21 @@ export const controllerLogLevel = {
7676
choices: ["trace", "debug", "info", "warn", "error"],
7777
} as Options,
7878
};
79+
80+
export const enableMetrics = {
81+
"enable-metrics": {
82+
description: "Enable Prometheus metrics server",
83+
type: "boolean",
84+
required: false,
85+
default: true,
86+
} as Options,
87+
};
88+
89+
export const metricsPort = {
90+
"metrics-port": {
91+
description: "Port for the Prometheus metrics server",
92+
type: "number",
93+
required: false,
94+
default: 9090,
95+
} as Options,
96+
};

‎pnpm-lock.yaml

+521-316
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)
Please sign in to comment.