hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zheng Shao (JIRA)" <j...@apache.org>
Subject [jira] Created: (HIVE-235) DynamicSerDe does not work with Thrift Protocols that can have missing fields for null values
Date Fri, 16 Jan 2009 11:23:59 GMT
DynamicSerDe does not work with Thrift Protocols that can have missing fields for null values
---------------------------------------------------------------------------------------------

                 Key: HIVE-235
                 URL: https://issues.apache.org/jira/browse/HIVE-235
             Project: Hadoop Hive
          Issue Type: Bug
          Components: Serializers/Deserializers
            Reporter: Zheng Shao
            Priority: Blocker


The current DynamicSerDe code assumes all fields are there and no fields are missing.

However Thrift Protocols can have missing fields, in case the field is null.

In that case, DynamicSerDe may commit 2 behavior:
1. array index out of bound error because DynamicSerDe assumes the number of fields in the
record should be equal to that in the DDL;
2. fields with null values will take the value from the last record. This may produce wrong
result for queries.

In order to fix this, we need to:
1. Pass ObjectInspector/TypeInfo recursively so that we know the number of fields when deserializing
the record.
2. Clear out fields that are missing from the record.


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message