hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-hudi] umehrot2 commented on a change in pull request #1330: [HUDI-607] Fix to allow creation/syncing of Hive tables partitioned by Date type columns
Date Mon, 17 Feb 2020 19:28:08 GMT
umehrot2 commented on a change in pull request #1330: [HUDI-607] Fix to allow creation/syncing
of Hive tables partitioned by Date type columns
URL: https://github.com/apache/incubator-hudi/pull/1330#discussion_r380339707
 
 

 ##########
 File path: hudi-spark/src/main/java/org/apache/hudi/DataSourceUtils.java
 ##########
 @@ -77,6 +80,11 @@ public static Object getNestedFieldVal(GenericRecord record, String fieldName,
b
 
       // return, if last part of name
       if (i == parts.length - 1) {
+
+        if (isLogicalTypeDate(valueNode, part)) {
 
 Review comment:
   This method seems to me like the right central place to perform this check and conversion.
If we check the accessors of this method, it is used by Hudi to retrieve the key values for
its metadata fields. It might be more standard to have Hudi treat Date as the actual Date
string instead of a Long, across all its keys for its internal usage. It would create a lot
of confusion otherwise and accessors of this functions will have to take care of adding this
check.
   
   Also, we would again have to re-write the same logic/loop again on client side to check
for logical type. Because this function basically returns only the value.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message