hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-hudi] umehrot2 commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema
Date Tue, 24 Mar 2020 10:10:49 GMT
umehrot2 commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields
if not present when rewriting incoming record with new schema
URL: https://github.com/apache/incubator-hudi/pull/1427#discussion_r397037244
 
 

 ##########
 File path: hudi-common/src/test/java/org/apache/hudi/common/util/TestHoodieAvroUtils.java
 ##########
 @@ -57,4 +60,16 @@ public void testPropsPresent() {
     }
     Assert.assertTrue("column pii_col doesn't show up", piiPresent);
   }
+
+  @Test
+  public void testDefaultValue() {
+    GenericRecord rec = new GenericData.Record(new Schema.Parser().parse(EXAMPLE_SCHEMA));
+    rec.put("_row_key", "key1");
+    rec.put("non_pii_col", "val1");
+    rec.put("pii_col", "val2");
+    rec.put("timestamp", 3.5);
 
 Review comment:
   Can you help me understand how you are running into this issue with default values ? 
   
   Based on my understanding, conversion to avro is internal to Hudi and a custom avro schema
(with default values) is not something that user can themselves pass. And how `spark-avro`
converts `struct schema to avro` there is no special handling there from `default value` perspective.
So I guess I am not sure whether this is an issue in the first place.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message