Test read write cs (Log scenario, random writes) #16999
Conversation
|
🟢 |
ydb/library/workload/log/log.cpp
Outdated
| @@ -252,6 +253,11 @@ class TRandomLogGenerator { | |||
| return result.str(); | |||
| } | |||
|
|
|||
| TInstant UniformInstant(ui64 from, ui64 to) const { | |||
| TMersenne<ui64> rnd; | |||
There was a problem hiding this comment.
локально проверил - если сид не передавать, то всегда одинаковые числа будут генериться.
Emgariko
force-pushed
the
test_log_scenario_random_writes
branch
from
208c01a to
61fb45b
Compare
Emgariko
force-pushed
the
test_log_scenario_random_writes
branch
from
61fb45b to
c58a8e9
Compare
|
⚪ Test history | Ya make output | Test bloat
⚪
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ Test history | Ya make output | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
| if timestamp_deviation is not None: | ||
| if date_from is not None: | ||
| raise TypeError("timestamp_deviation and uniform PK mode should be applied mutual exclusively") | ||
| if date_to is not None: | ||
| raise TypeError("timestamp_deviation and uniform PK mode should be applied mutual exclusively") | ||
| self.timestamp_deviation = timestamp_deviation | ||
| self.pk_mode = YdbWorkloadLog.PKMode.TIMESTAMP_DEVIATION | ||
| else: | ||
| if date_from is not None: | ||
| if timestamp_deviation is not None: | ||
| raise TypeError("timestamp_deviation and uniform PK mode should be applied mutual exclusively") | ||
| if date_to is None: | ||
| raise TypeError("missing date_to") | ||
| self.date_from = date_from | ||
| self.date_to = date_to | ||
| self.pk_mode = YdbWorkloadLog.PKMode.UNIFORM | ||
| else: | ||
| raise TypeError("missing date_from") |
There was a problem hiding this comment.
Уродливо получилось, надо поправить
Emgariko
force-pushed
the
test_log_scenario_random_writes
branch
from
79b23db to
b9b5771
Compare
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
| @@ -153,3 +184,35 @@ def test(self, timestamp_deviation: int): | |||
|
|
|||
| for thread in threads: | |||
| thread.join() | |||
|
|
|||
| def test_log_uniform(self): | |||
There was a problem hiding this comment.
При локальном прогоне тест TestLogScenario(содержащий в себе 3 теста - test_log_deviation с двумя pytest-параметрами + test_log_uniform) почему-то считает общее время для этих трех тестов. Время превыщает 600 секунд и падает по таймауту.
Ограничение по памяти тоже превышается. Вероятно по этой же причине.
Скорее всего тесты надо разделить или как-то поправить конфигурацию, чтобы считалось не суммарное время 3х тестов, а каждого независимо.
ydb/tests/olap <py3test> [size:medium]
│------ sole chunk ran 3 tests (total:621.35s - test:600.04s)
│Chunk exceeded 600s timeout and was killed
│List of the tests involved in the launch:
│test_log_scenario.py::TestLogScenario::test_log_deviation[1051200] (good) duration: 282.46s
│test_log_scenario.py::TestLogScenario::test_log_uniform (timeout) duration: 212.66s
│test_log_scenario.py::TestLogScenario::test_log_deviation[180] (good) duration: 121.21s
│Info: Test run has exceeded 9.0G (9437184K) memory limit with 14.0G (14646960K) used. This may lead to test failure on the Autocheck/CI
There was a problem hiding this comment.
Попробую покопать в сторону FORK_SUBTESTS().
Смущает, что эта опция применяется для всех тестов в директории. Но должна решить мою проблему.
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ Test history | Ya make output | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
| "timestamp uniformly from specified interval. Presents as seconds since epoch. Once this option passed, 'date-from' " | ||
| "should be passed as well. This option is mutually exclusive with 'timestamp_deviation'") | ||
| .Optional().StoreResult(&TimestampDateTo); | ||
|
|
There was a problem hiding this comment.
MutuallyExclusive на timestamp_deviation и date_{to, from} потерял когда мержил конфликты.
Changelog entry
Implementing #13531
Changelog category
Description for reviewers
Added
date_from&date_toparams toydb workload logto support uniformly distributed timestamp generation. And implemented test for random writes scenario.