Test cs read-write. Log scenario (write in the end) #16018
Conversation
|
🟢 |
Emgariko
force-pushed
the
test_log_scenario_1
branch
3 times, most recently
from
ddbac88 to
237881b
Compare
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
ydb/tests/olap/ttl_tiering/base.py
Outdated
| class S3Client: | ||
| def __init__(self, endpoint, region, key_id, key_secret): | ||
| self.endpoint = endpoint | ||
| self.region = region | ||
| self.key_id = key_id | ||
| self.key_secret = key_secret | ||
|
|
||
| session = boto3.session.Session() | ||
| self.s3 = session.resource( | ||
| service_name="s3", | ||
| aws_access_key_id=key_id, | ||
| aws_secret_access_key=key_secret, | ||
| region_name=region, | ||
| endpoint_url=endpoint | ||
| ) | ||
| self.client = session.client( | ||
| service_name="s3", | ||
| aws_access_key_id=key_id, | ||
| aws_secret_access_key=key_secret, | ||
| region_name=region, | ||
| endpoint_url=endpoint | ||
| ) | ||
|
|
||
| def create_bucket(self, name: str): | ||
| self.client.create_bucket(Bucket=name) | ||
|
|
||
| def get_bucket_stat(self, bucket_name: str) -> (int, int): | ||
| bucket = self.s3.Bucket(bucket_name) | ||
| count = 0 | ||
| size = 0 | ||
| for obj in bucket.objects.all(): | ||
| count += 1 | ||
| size += obj.size | ||
| return (count, size) | ||
|
|
||
|
|
||
| class ColumnTableHelper: | ||
| def __init__(self, ydb_client: YdbClient, path: str): | ||
| self.ydb_client = ydb_client | ||
| self.path = path | ||
|
|
||
| def get_row_count(self) -> int: | ||
| return self.ydb_client.query(f"select count(*) as Rows from `{self.path}`")[0].rows[0]["Rows"] | ||
|
|
||
| def get_portion_count(self) -> int: | ||
| return self.ydb_client.query(f"select count(*) as Rows from `{self.path}/.sys/primary_index_portion_stats`")[0].rows[0]["Rows"] | ||
|
|
||
| def get_portion_stat_by_tier(self) -> dict[str, dict[str, int]]: | ||
| results = self.ydb_client.query(f"select TierName, sum(Rows) as Rows, count(*) as Portions from `{self.path}/.sys/primary_index_portion_stats` group by TierName") | ||
| return {row["TierName"]: {"Rows": row["Rows"], "Portions": row["Portions"]} for result_set in results for row in result_set.rows} | ||
|
|
||
| def get_blob_stat_by_tier(self) -> dict[str, (int, int)]: | ||
| stmt = f""" | ||
| select TierName, count(*) as Portions, sum(BlobSize) as BlobSize, sum(BlobCount) as BlobCount from ( | ||
| select TabletId, PortionId, TierName, sum(BlobRangeSize) as BlobSize, count(*) as BlobCount from `{self.path}/.sys/primary_index_stats` group by TabletId, PortionId, TierName | ||
| ) group by TierName | ||
| """ | ||
| results = self.ydb_client.query(stmt) | ||
| return {row["TierName"]: {"Portions": row["Portions"], "BlobSize": row["BlobSize"], "BlobCount": row["BlobCount"]} for result_set in results for row in result_set.rows} | ||
|
|
||
|
|
There was a problem hiding this comment.
Вот это я не понял откуда пришло, видимо с коммитов Влада
|
План был такой: |
ydb/tests/olap/ya.make
Outdated
| REQUIREMENTS(ram:25) | ||
|
|
||
| SIZE(LARGE) | ||
| TAG(ya:fat) |
There was a problem hiding this comment.
ram поднял потому что при прогоне на дев машинке превышался лимит.
LARGE сделал потому что прогон занимает 600+ секунд
There was a problem hiding this comment.
Поменяю параметры теста чтобы прогон был быстрее. Тогда оставлю Medium
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
Нужно будет размьютить тест |
Emgariko
force-pushed
the
test_log_scenario_1
branch
2 times, most recently
from
790de24 to
0d66f11
Compare
Emgariko
force-pushed
the
test_log_scenario_1
branch
from
0d66f11 to
13f0c42
Compare
|
Там еще кажется папка с тестами была замьючена, надо размьютить |
|
⚪ Test history | Ya make output | Test bloat
⚪
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ Test history | Ya make output | Test bloat
⚪
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
Emgariko
changed the title
Changelog entry
Test #13530
Changelog category
Description for reviewers
...