Skip to content

DLT Sink API integration #157

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 20 commits into from
Mar 26, 2025
Merged

DLT Sink API integration #157

merged 20 commits into from
Mar 26, 2025

Conversation

ravi-databricks
Copy link
Contributor

  • Added support for dlt.sink api
  • supports delta, kafka and eventshub

@ravi-databricks ravi-databricks self-assigned this Feb 28, 2025
@ravi-databricks ravi-databricks added the enhancement New feature or request label Feb 28, 2025
@ravi-databricks ravi-databricks added this to the v0.0.10 milestone Feb 28, 2025
@ravi-databricks ravi-databricks linked an issue Feb 28, 2025 that may be closed by this pull request
Add non-Delta as Sink #48
Open
brij-raghuwanshi-db
brij-raghuwanshi-db reviewed Mar 26, 2025
View reviewed changes
Copy link

@brij-raghuwanshi-db brij-raghuwanshi-db left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check for hardcoding and for the version updates in different locations.
Can we create one file called "version.py" and import it at all the other locations.

integration_tests/conf/cloudfiles-onboarding.template
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need hardcoding here?
"tableName": "ravi_dlt_meta_uc.dlt_meta_bronze_it_e33f69c0b2824f62b651a3d047e30c31.customer_sink_cdc"

setup.py
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a single location to ensure that the version number is updated at all places?
Currently, we have:

  • setup.py
  • src/about.py
  • src/init.py

src/__about__.py
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a single location to ensure that the version number is updated at all places?
Currently, we have:

setup.py
src/about.py
src/init.py

src/__init__.py
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a single location to ensure that the version number is updated at all places?
Currently, we have:

setup.py
src/about.py
src/init.py

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These all will be changed as part of repo package restructuring under issue_159

@brij-raghuwanshi-db brij-raghuwanshi-db self-requested a review March 26, 2025 17:53
brij-raghuwanshi-db
brij-raghuwanshi-db approved these changes Mar 26, 2025
View reviewed changes
Copy link

@brij-raghuwanshi-db brij-raghuwanshi-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ravi-databricks ravi-databricks merged commit 1ea6f4c into feature/v0.0.10 Mar 26, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@brij-raghuwanshi-db brij-raghuwanshi-db brij-raghuwanshi-db approved these changes

@ganeshchand ganeshchand Awaiting requested review from ganeshchand

Assignees

@ravi-databricks ravi-databricks

Labels
enhancement New feature or request
Projects
None yet
Milestone
v0.0.10
Development

Successfully merging this pull request may close these issues.

Add non-Delta as Sink
3 participants
@ravi-databricks @brij-raghuwanshi-db @harsha-pasala