hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sreenath Menon <>
Subject Compressed data storage in HDFS - Error
Date Wed, 06 Jun 2012 08:50:23 GMT
I would like to compress my data in the HDFS using some Hive commands.
Step followed: (data already residing in table sample)

create table rc_lzo like sample;
SET hive.exec.compress.output=true;
SET mapred.output.compression.codec=com.hadoop.compression.lzo.LzoCodec;
insert overwrite table rc_lzo select * from sample;

Compression codec com\.hadoop\.compression\.lzo\.LzoCodec was not found

1)What do I need to do to use Lzo as well as other compression methods?

2)Heard somewhere that :Using compressed data will produce better results
than uncompressed data in some cases. How can this be, as there is always a
compression and decompression time allotted with compression methods. Any
truth in this, if so how ? Can understand how there are better results when
using compression between mappers-to-reducers and in between map-reduce

Thanks and Regards
Sreenath Mullassery

View raw message