hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuntao Jia (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HIVE-640) Add LazyBinarySerDe to Hive
Date Wed, 29 Jul 2009 05:28:14 GMT

     [ https://issues.apache.org/jira/browse/HIVE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yuntao Jia updated HIVE-640:
----------------------------

    Attachment: HIVE-640.1.patch

The first version patch. The major changes are:

1/ Added the LazyBinarySerDe class. 

2/ Added package "org.apache.hadoop.hive.serde2.lazybinary" which include the LazyBinary classes
for all Hive supported data types, such as Boolean, Byte, Short, Integer, Long, Float, Double,
String, Array, Struct and Map. It also include a LazyBinaryFactory class and a utility function
class.

3/ Added package "org.apache.hadoop.hive.serde2.lazybinary.objectinspector" which include
the object inspector classes for non-primitive data types, such as List, Struct and Map. It
also includes an object inspector factory class. For primitive data types, such as Boolean,
Byte, Short and so on, I used the corresponding writable object inspector classes. For instance,
I used WritableBooleanObjectInspector for LazyBinaryBoolean. 

4/ Added a unit test for LazyBinarySerDe. It tests serializatoin and deserialization of all
above data types except LazyBinaryMap. I will include a test for that in the future.


> Add LazyBinarySerDe to Hive
> ---------------------------
>
>                 Key: HIVE-640
>                 URL: https://issues.apache.org/jira/browse/HIVE-640
>             Project: Hadoop Hive
>          Issue Type: New Feature
>            Reporter: Zheng Shao
>            Assignee: Yuntao Jia
>         Attachments: HIVE-640.1.patch
>
>
> LazyBinarySerDe will serialize the data in binary format while supporting LazyDeserialization.
> This will be used as the SerDe for value between map and reduce, and also between different
map-reduce jobs.
> This will help improve the performance of Hive a lot.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message