hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "He Yongqiang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-956) Add support of columnar binary serde
Date Wed, 08 Jun 2011 08:39:58 GMT

    [ https://issues.apache.org/jira/browse/HIVE-956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045839#comment-13045839
] 

He Yongqiang commented on HIVE-956:
-----------------------------------

i got one conflict when trying to apply this patch to my local. Can you rebase?

a few minor comments:
1. is getLength() only for null check? if yes, can you call it 'isNull()'? And if the only
difference in ColumnarStruct and BinaryColumnarStruct is null check, just curious, how difficult
is it to avoid this new BinaryColumnarStruct class?
2. can you add a 'toString()' for new binary columnar serde, just the same as columnar serde
3. why do you need to handle empty string specially? "serializeStream.write(INVALID_UTF__SINGLE_BYTE,
0, 1);" i thought for empty data, we just store data length 0 in rcfile.
I thought since there is no NULLSequence in the new serde, the null should be handled specially.
 i am missing sth, How do you handle null here?
4. can LazyBinaryColumnarSerDe share some code with LazyBinarySerde?
5. how warnedOnceNullMapKey is used? 
originally the map comparison is not supported, but this patch added a mapEqualComparer. can
we put this in a separate patch? It seems the logic in CrossMapEqualComparer is not correct.
(how do you make sure you will get the keys from a map in some kind of same order?)


Thanks for this great work! Pls resubmit the patch after rebase


> Add support of columnar binary serde
> ------------------------------------
>
>                 Key: HIVE-956
>                 URL: https://issues.apache.org/jira/browse/HIVE-956
>             Project: Hive
>          Issue Type: New Feature
>            Reporter: He Yongqiang
>            Assignee: Krishna Kumar
>         Attachments: HIVE.956.patch.0, HIVE.956.patch.1
>
>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message