hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mariappan Asokan (JIRA)" <>
Subject [jira] [Updated] (HIVE-8355) Compress serialized bytes for all objects before encoding
Date Fri, 10 Oct 2014 01:42:34 GMT


Mariappan Asokan updated HIVE-8355:
    Attachment: HIVE-8355.patch

Modified {{HCatUtil.serialize()}} and {{HCatUtil.deserialize()}} to do compression and decompression
respectively.  Reverted HCATALOG-453 changes in {{InputJobInfo}} since the whole serialized
instance of it will be compressed not just the data member {{partitions}}.

> Compress serialized bytes for all objects before encoding
> ---------------------------------------------------------
>                 Key: HIVE-8355
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: HCatalog
>    Affects Versions: 0.13.1
>            Reporter: Mariappan Asokan
>         Attachments: HIVE-8355.patch
> The {{HCatUtil.serialize()}} method can compress the serialized bytes before encoding
to cut down the number of bytes shipped via Hadoop Configuration object since there is a limit
of about 5 MB (see HCATALOG-453.)  The complementary method {{HCatUtil.deserialize()}} should
decode, decompress, and deserialize.

This message was sent by Atlassian JIRA

View raw message