spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [spark] MaxGekk commented on issue #26134: [SPARK-29486][SQL] CalendarInterval should have 3 fields: months, days and microseconds
Date Thu, 17 Oct 2019 18:15:43 GMT
MaxGekk commented on issue #26134: [SPARK-29486][SQL] CalendarInterval should have 3 fields:
months, days and microseconds
URL: https://github.com/apache/spark/pull/26134#issuecomment-543298160
 
 
   > I'm really on the fence. @MaxGekk do you have some better ideas?
   
   I would strictly follow the SQL standard and implement SQL `TIMESTAMP WITHOUT TZ` and `TIMESTAMP
WITH TZ` (with TZ_HOUR:TZ_MINUTE). This could be done when `spark.sql.ansi.enabled = true`,
or independently from the setting.
   
   For our default settings at the moment `spark.sql.ansi.enabled = false`, I would **not**
change the session local time zone because we definitely break existing user apps. Changing
structure of `INTERVAL` type looks reasonable. 
   
   > To be honest I'd like to make Spark follow SQL standard and use TIMESTAMP WITHOUT
TIMEZONE, but it changes the semantic and we don't know how to deal with existing timestamp
data.
   
   Spark's `TIMESTAMP` and SQL `TIMESTAMP WITHOUT TIMEZONE` is different types, no doubt.
I don't think we can smoothly replace one by another one. I imagine the types as similar Java
8 classes:
   Spark's `TIMESTAMP` - `java.time.Instant`
   SQL `TIMESTAMP WITHOUT TZ` - `java.time.LocalDateTime`
   SQL `TIMESTAMP WITH TZ` - `java.time.ZonedDateTime`
   
   As Java 8 time API has all 3 types, like that Spark could have all 3.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message