hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <>
Subject [jira] [Created] (HIVE-16611) Kryo remove field is not working
Date Tue, 09 May 2017 00:03:04 GMT
Prasanth Jayachandran created HIVE-16611:

             Summary: Kryo remove field is not working
                 Key: HIVE-16611
             Project: Hive
          Issue Type: Bug
          Components: Serializers/Deserializers
    Affects Versions: 3.0.0
            Reporter: Prasanth Jayachandran

I have seen some instances where removeField in SerializationUtilities is not working
      removeField(kryo, Operator.class, "colExprMap");
      removeField(kryo, AbstractOperatorDesc.class, "statistics");

These 2 fields are not expected to be serialized but some stacktraces where custom UDFs are
involved these fields are observed in the stacktrace. 

genericUDF (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc) 
chidren (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc) 
colExprMap (org.apache.hadoop.hive.ql.exec.SelectOperator) 
childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator) 
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork) 

at org.apache.hadoop.hive.ql.exec.Utilities.deserializeObjectByKryo( 
at org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan( 
at org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan( 

This needs further investigation. 

This message was sent by Atlassian JIRA

View raw message