hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joydeep Sen Sarma (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-270) Add a lazy-deserialized SerDe for space and cpu efficient serialization of rows with primitive types
Date Mon, 16 Feb 2009 02:40:59 GMT

    [ https://issues.apache.org/jira/browse/HIVE-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12673776#action_12673776
] 

Joydeep Sen Sarma commented on HIVE-270:
----------------------------------------

a couple of things i missed in first review:

- where we get numberformatexception in lazy number parsing - should we return serdeexception
instead? I am just worried that by sending them across as nulls - we are totally hiding any
errors from the user.

  right now the serdeexceptions just go into a hadoop counter - but perhaps later we can surface
them to the user (and error out job on threshold number of exceptions)

- can we replace all occurences of metadatatypedserde? it seems that there are quite a few
places in the code that are still using it (see planutils.getdefaulttabledesc for instance
- that goes into a bunch of places - like script operator, fetch task, filesink)

> Add a lazy-deserialized SerDe for space and cpu efficient serialization of rows with
primitive types
> ----------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-270
>                 URL: https://issues.apache.org/jira/browse/HIVE-270
>             Project: Hadoop Hive
>          Issue Type: New Feature
>          Components: Serializers/Deserializers
>            Reporter: Zheng Shao
>            Assignee: Zheng Shao
>         Attachments: HIVE-270.1.patch, HIVE-270.3.patch, HIVE-270.4.patch, HIVE-270.5.patch
>
>
> We want to add a lazy-deserialized SerDe for space and cpu efficient serialization of
rows with primitive types.
> This SerDe will share the same format as MetadataTypedColumnsetSerDe/TCTLSeparatedProtocol
to be backward compatible.
> This SerDe will be used to replace the default table SerDe, and the SerDe used to communicate
with user scripts.
> For simplicity, we don't plan to support nested structure with this SerDe.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message