Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-12580. Set up Temporary RocksDB for Storing Container Log Information #8072

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

sreejasahithi
Copy link
Contributor

@sreejasahithi sreejasahithi commented Mar 13, 2025

What changes were proposed in this pull request?

This change introduces the setup of temporary RocksDB storage and creates two key tables to store container log data for debugging purposes within Ozone. Also implemented codec class to serialize and deserialize data for both tables.

The two tables are as follows:

1. Container Log Table (ContainerLogTable):
        Key: Container ID
        Value: A list of ContainerInfo objects that store:
        Container's final state
        Datanode ID
        Final BCSID

2.   Datanode Container Log Table (DatanodeContainerLogTable):
        Key: Container ID and Datanode ID combination
        Value: A list of DatanodeContainerInfo objects containing:
        Timestamp of the state transition
        Container state (e.g., OPEN, CLOSED, CLOSING, QUASI_CLOSED, UNHEALTHY, DELETED ....)
        BCSID
        Error message (optional)

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-12580

How was this patch tested?

Tested this locally with sample log files.

Below is the workflow link that passed successfully:
https://github.com/sreejasahithi/ozone/actions/runs/13832960621

@errose28
Copy link
Contributor

Thanks for working on this @sreejasahithi. Would it be better to use a SQL DB here instead of RocksDB? dn-container.log files are already row and column based, so they easily map to a tabular format. A SQL DB would give us much more flexibility with our queries, while with RocksDB we would be limited to whatever wrappers we create to work with the binary value formats.

@sreejasahithi sreejasahithi marked this pull request as ready for review March 14, 2025 04:17
@sreejasahithi
Copy link
Contributor Author

@errose28 , Thanks for the suggestion! I'll give it a try with SQLite and see how it performs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants