hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema
Date Tue, 24 Mar 2020 09:00:47 GMT
pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy default values
of fields if not present when rewriting incoming record with new schema
URL: https://github.com/apache/incubator-hudi/pull/1427#discussion_r396992378
 
 

 ##########
 File path: hudi-common/src/test/java/org/apache/hudi/common/util/TestHoodieAvroUtils.java
 ##########
 @@ -57,4 +60,16 @@ public void testPropsPresent() {
     }
     Assert.assertTrue("column pii_col doesn't show up", piiPresent);
   }
+
+  @Test
+  public void testDefaultValue() {
+    GenericRecord rec = new GenericData.Record(new Schema.Parser().parse(EXAMPLE_SCHEMA));
+    rec.put("_row_key", "key1");
+    rec.put("non_pii_col", "val1");
+    rec.put("pii_col", "val2");
+    rec.put("timestamp", 3.5);
 
 Review comment:
   No its not that the original record has default values as null. Its just while getting
the values from the record, default values are not considered. Please have a look at this
function from avro-1.8.2 library - 
   
   @Override public Object get(String key) {
         Field field = schema.getField(key);
         if (field == null) return null;
         return values[field.pos()];
       }
   
   Ideally the above function should return field.defaultVal() in case values[field.pos()]
is null, but that is not the case. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message