hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zheng Shao (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-337) LazySimpleSerDe should support array and map types
Date Wed, 11 Mar 2009 22:18:50 GMT

    [ https://issues.apache.org/jira/browse/HIVE-337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12681064#action_12681064

Zheng Shao commented on HIVE-337:

For the migration path, we can easily create new tables with new SerDes, and everything will
work transparently. If you are suggesting letting LazySerDe automatically figure out the old/new
format, I don't think that's even possible, and if it is, users will be easily confused by

Delimited format is meant to be simple and human-readable, and it is only good for simple
data. If the structure really gets complicated, we should store the data in binary format
instead of delimited format. For example, we can use Thrift etc.

If we really want to write a new SerDe that shares a lot with LazySimpleSerDe (with an extended
delimited format), we can easily do that by reusing a lot of the classes introduced by LazySimpleSerDe.
There is not much to copy - and if there is, it's better to factor the common code out, instead
of pushing all logics (new format/old format) into the same class.

Let's open another jira for discussions on new features like this.

So the question here is that we have to make a choice between the two: whether treat "" to
be an empty array or an array with an empty string as the only element (and the same question
for NULL).

> LazySimpleSerDe should support array and map types
> --------------------------------------------------
>                 Key: HIVE-337
>                 URL: https://issues.apache.org/jira/browse/HIVE-337
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Serializers/Deserializers
>    Affects Versions: 0.2.0
>            Reporter: Zheng Shao
>            Assignee: Zheng Shao
>            Priority: Blocker
> Once we do that, we can completely deprecate DynamicSerDe/TCTLSeparatedProtocol, and
close any bugs that DynamicSerDe/TCTLSeparatedProtocol has.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message