hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bharath vissapragada <bharathvissapragada1...@gmail.com>
Subject Re: Setting up Snappy compression in Hadoop
Date Wed, 01 Jan 2014 16:37:21 GMT
Did you build it for your platform? You can do an "ldd" on the .so file to
check if the dependent libs are present. Also make sure you placed it in
the right directory for your platform (Linux-amd64-64 or Linux-i386-32)


On Wed, Jan 1, 2014 at 10:02 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Please take a look at http://hbase.apache.org/book.html#snappy.compression
>
> Cheers
>
>
> On Wed, Jan 1, 2014 at 8:05 AM, Amit Sela <amits@infolinks.com> wrote:
>
>> Hi all,
>>
>> I'm running on Hadoop 1.0.4 and I'd like to use Snappy for map output
>> compression.
>> I'm adding the configurations:
>>
>> configuration.setBoolean("mapred.compress.map.output", true);
>> configuration.set("mapred.map.output.compression.codec",
>> "org.apache.hadoop.io.compress.SnappyCodec");
>>
>> And I've added libsnappy.so.1 to $HADOOP_HOME/lib/native/Linux-amd64-64/
>>
>> Still, all map tasks fail with "native snappy library not available".
>>
>> Could anyone elaborate on how to install Snappy for Hadoop ?
>>
>> Thanks,
>>
>> Amit.
>>
>
>

Mime
View raw message