hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frank Grimes <frankgrime...@gmail.com>
Subject Can spill to disk be in compressed format to reduce I/O?
Date Thu, 12 Jan 2012 15:40:24 GMT
Hi All,

We're trying to speed up an M/R job which combines multiple .avro files.
We've noticed that when it spills to disk, it's in uncompressed format.
Is there a way to make it spill temporary segments as .avro with Deflate compression?


Frank Grimes
View raw message