hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Debarshi Basak <debarshi.ba...@tcs.com>
Subject Re: Compressed data storage in HDFS - Error
Date Wed, 06 Jun 2012 09:19:46 GMT
<font face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size="2">Basically,
when your data is compressed you have lesser IO than your uncompressd data. During job execution
is doesn't decompress. It would be a relevant question in Hadoop's mailing list than hive.<BR><BR><BR>Debarshi
Basak<BR>Tata Consultancy Services<BR>Mailto: debarshi.basak@tcs.com<BR>Website:
http://www.tcs.com<BR>____________________________________________<BR>Experience
certainty. IT Services<BR>Business Solutions<BR>Outsourcing<BR>____________________________________________<BR><BR><FONT
color=#990099>-----Sreenath Menon <SREENATHMENON5@GMAIL.COM>wrote: -----</FONT>

<DIV>
<BLOCKQUOTE style="PADDING-RIGHT: 0px; PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT:
black 2px solid; MARGIN-RIGHT: 0px">To: user@hive.apache.org<BR>From: Sreenath Menon
&lt;sreenathmenon5@gmail.com&gt;<BR>Date: 06/06/2012 02:20PM<BR>Subject:
Compressed data storage in HDFS - Error<BR><BR>I would like to compress my data
in the HDFS using some Hive commands.<BR>Step followed: (data already residing in table
sample)<BR><BR>create table rc_lzo like sample;<BR>SET hive.exec.compress.output=true;<BR>SET
mapred.output.compression.codec=com.hadoop.compression.lzo.LzoCodec;<BR>insert overwrite
table rc_lzo select * from sample;<BR><BR>Error:<BR>Compression codec com\.hadoop\.compression\.lzo\.LzoCodec
was not found<BR><BR>1)What do I need to do to use Lzo as well as other compression
methods?<BR><BR>2)Heard somewhere that :Using compressed data will produce better
results than uncompressed data in some cases. How can this be, as there is always a compression
and decompression time allotted with compression methods. Any truth in this, if so how ? Can
understand how there are better results when using compression between mappers-to-reducers
and in between map-reduce jobs.<BR><BR>Thanks and Regards<BR>Sreenath Mullassery<BR></BLOCKQUOTE></DIV>
<DIV></DIV></font><p>=====-----=====-----=====<br>
Notice: The information contained in this e-mail<br>
message and/or attachments to it may contain <br>
confidential or privileged information. If you are <br>
not the intended recipient, any dissemination, use, <br>
review, distribution, printing or copying of the <br>
information contained in this e-mail message <br>
and/or attachments to it are strictly prohibited. If <br>
you have received this communication in error, <br>
please notify us by reply e-mail or telephone and <br>
immediately and permanently delete the message <br>
and any attachments. Thank you</p>

<p></p>

Mime
View raw message