You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
CONVERT TO DELTA fails with Syntax Error when we add a TIMESTAMP column in the PARTITIONED BY clause. When I change it to date it works.
Steps to reproduce
Copy parquet Files produced by Spark .write.format("parquet").partitionBy on a timestamp column into a folder in AWS S3. eg parquet_path/event_time_partition=2025-02-19 13%3A00%3A00
Run in spark.sql with Delta configurations.
CONVERT TO DELTA parquet.`s3://bucket/path/to/parquet_path` PARTITIONED BY ( event_timestamp timestamp)
Change timestamp to date, it works(not intended)
CONVERT TO DELTA parquet.`s3://bucket/path/to/parquet_path` PARTITIONED BY ( event_timestamp date)
Observed results
ParseException:
[PARSE_SYNTAX_ERROR] Syntax error at or near 'CONVERT'.(line 1, pos 0)
== SQL ==
CONVERT TO DELTA parquet.`s3://bucket/path/to/parquet_path` PARTITIONED BY ( event_timestamp timestamp)
^^^
Expected results
I want to know if this is unsupported. If yes, it should be correctly indicated. Now it is confusing. Also, If I change timestamp to date type it works but that is not the expectation.
Further details
Environment information
Delta Lake version: 3.2.0
Spark version: 3.5.1
Scala version: 2.12
Willingness to contribute
The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?
Yes. I can contribute a fix for this bug independently.
Yes. I would be willing to contribute a fix for this bug with guidance from the Delta Lake community.
No. I cannot contribute a bug fix at this time.
The text was updated successfully, but these errors were encountered:
Bug
Which Delta project/connector is this regarding?
Describe the problem
CONVERT TO DELTA fails with Syntax Error when we add a TIMESTAMP column in the PARTITIONED BY clause. When I change it to date it works.
Steps to reproduce
.write.format("parquet").partitionBy
on a timestamp column into a folder in AWS S3. egparquet_path/event_time_partition=2025-02-19 13%3A00%3A00
timestamp
todate
, it works(not intended)Observed results
Expected results
I want to know if this is unsupported. If yes, it should be correctly indicated. Now it is confusing. Also, If I change timestamp to
date
type it works but that is not the expectation.Further details
Environment information
Willingness to contribute
The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?
The text was updated successfully, but these errors were encountered: