Support the SUPER type for JSONB type in the source [LOYAL-10940]

[quocnguyendinh]

https://kaligo.atlassian.net/browse/LOYAL-10940

Background

• We’re replicating Postgres JSONB as Redshift VARCHAR and convert them at staging layer
• By default, we need to convert VARCHAR to SUPER type in Redshift via PARSE_JSON(...). Instead, we can move this from T to EL part (embedded into Target-Redshift).

Design

Tune the codebase in order to enable convert Postgres JSONB column into Redshift SUPER type.

Impact

No need to worry about new tables with JSONB columns and also save more resource since we no longer use PARSE_JSON(...) on top of view in the original way.

Caveats

None

Testing

• Setup the Python Virtual Environment based on the requirements.txt
• Run coverage run -m pytest -vv --disable-pytest-warnings tests/unit && coverage report to check the unit tests:
  

Docs

None

[khoaanguyenn]

💯