spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <>
Subject [GitHub] [spark] huaxingao commented on pull request #29043: [SPARK-32205][SQL] Writing timestamp to mysql should be datetime type
Date Sun, 12 Jul 2020 05:56:52 GMT

huaxingao commented on pull request #29043:

   I am still not convinced that it's a good idea to map TimeStamp to DateTime in MySQLDialect.
Using an example similar to yours:
   sql("select cast('1970-01-01 00:00:01' as timestamp)").toDF("ts").write.mode("append").jdbc("jdbc:mysql://localhost:3306/test",
   Since user explicitly cast string to timestamp, I would think that the user wants to insert
1970-01-0 00:00:01 as a TimeStamp data type. Suppose the current time zone on mysql server
is america/los_angeles. After the data is inserted to mysql, if the user changes the time
zone setting
   SET TIME_ZONE = "america/new_york";
   Then the user would expect to get 1970-01-01 03:00:01 when retrieving the timestamp. But
with this patch, we silently change the data type from TimeStamp to DateTime, after user insert
the timestamp and change the time zone setting, user will still get 1970-01-0 00:00:01 which
is not the correct value.

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message