Low performance of queries with filters in YDB OLAP database

[vitalyisaev2]

The Cloud Logging connector performs parallel reading from three YDB OLAP databases located in different data centers. Queries containing timestamp filters catn take a very long time to complete. The query latency is between 1 sec and 7-8 minutes. The slowest responses are observed in VLA, however SAS and KLG can be slow as well.

Here is the list of Cloud Logging databases in pre-production environment:

• /pre-prod_vla/yc.logs.cloud/cc8jliaf18k2b9ae2bio
• /pre-prod_klg/yc.logs.cloud/cc817gnjjceirp80h587
• /pre-prod_sas/yc.logs.cloud/cc8bajrmntk9q0d1lc0t

Example of YQ query executed against the Cloud Logging connection:

SELECT COUNT(*) FROM cloud_logging_connection.`cloud-trail` WHERE
    timestamp >= Timestamp("2025-03-11T18:00:00Z")
    AND
    timestamp <= Timestamp("2025-03-11T18:00:01Z");

The query above is transformed by Connector into the following YDB query:

PRAGMA TablePathPrefix("/pre-prod_klg/yc.logs.cloud/cc817gnjjceirp80h587");
DECLARE $p0 AS Timestamp;
DECLARE $p1 AS Bool;
DECLARE $p2 AS Timestamp;
DECLARE $p3 AS Bool;
SELECT `timestamp` FROM `logs/origin/aoeoqusjtbo4m549jrom/aoe3cidh5dfee2s6cqu5/af3731rdp83d8gd8fjcv` WHERE (COALESCE((`timestamp` >= $p0), $p1) AND COALESCE((`timestamp` <= $p2), $p3))

Query args: Arguments: ["2025-03-11T18:00:00Z", false, "2025-03-11T18:00:01Z", false]

[dorooleg]

В итоге выяснили что не работает push down через ReadRanges для COALESCE с параметром:
Явно проставлен второй аргумент в false:

Через параметры:

[dorooleg]

Из странного сам фильтр разложился в правильный range

[dorooleg]

Еще нужно проверить что если не смогли сделать push down для ReadRanges, то на самом деле его можно завернуть в SSA программу с помощью которой можно пройтись по метаднным и отфильтровать данные