hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mariappan Asokan" <maso...@syncsort.com>
Subject Re: Review Request 26536: Compress serialized bytes for all objects before encoding.
Date Fri, 10 Oct 2014 16:02:56 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26536/
-----------------------------------------------------------

(Updated Oct. 10, 2014, 4:02 p.m.)


Review request for hcatalog and hive.


Changes
-------

Removed imports that are not needed anymore from InputJobInfo.java


Repository: hive-git


Description
-------

Currently all objects shipped via Hadoop Configuration are not compressed.  Compression is
limited to the data member "partitions" in InputJobInfo.  This patch will compress all serialized
objects.  This will cut down the traffic in the cluster when Configuration object is shipped
to all compute nodes.  Also, there is a 5 MB size limit on the Configuration object.  Compression
will make this limiit hardly reachable.


Diffs (updated)
-----

  hcatalog/core/src/main/java/org/apache/hive/hcatalog/common/HCatUtil.java 4fdb5c9 
  hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/InputJobInfo.java 360e77b


Diff: https://reviews.apache.org/r/26536/diff/


Testing
-------

No new test required.  All existing tests should pass.


Thanks,

Mariappan Asokan


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message