Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8E349101F6 for ; Wed, 1 Jan 2014 16:05:55 +0000 (UTC) Received: (qmail 37368 invoked by uid 500); 1 Jan 2014 16:05:44 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 37066 invoked by uid 500); 1 Jan 2014 16:05:40 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 37057 invoked by uid 99); 1 Jan 2014 16:05:39 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Jan 2014 16:05:39 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of amits@infolinks.com designates 207.126.144.121 as permitted sender) Received: from [207.126.144.121] (HELO eu1sys200aog106.obsmtp.com) (207.126.144.121) by apache.org (qpsmtpd/0.29) with SMTP; Wed, 01 Jan 2014 16:05:33 +0000 Received: from mail-ie0-f174.google.com ([209.85.223.174]) (using TLSv1) by eu1sys200aob106.postini.com ([207.126.147.11]) with SMTP ID DSNKUsQ8t8ju9Rj1APM5jYHLlZoPQXiUbhY5@postini.com; Wed, 01 Jan 2014 16:05:12 UTC Received: by mail-ie0-f174.google.com with SMTP id at1so13985190iec.33 for ; Wed, 01 Jan 2014 08:05:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=cjaAT2dN/tW+k+rQo1rVeogIsG2v6Fq+4yGFfSTNShM=; b=f6+TW/Hq6tW5xqPjx4Nn+n35MJNBmNQNlYRGvNb7llxzTGCGGm6EDYF8CGcZ1JlXD5 QZpT7agYRh7bly6WRYZTQ8OFIxCNr0LvJkIxqz9GqO6sXP05+w73iXEmCJM3Ciy3eb+m A9j71UrCbcZFvG4D9svIjOcVV1UVzl2Qa+/gmgfprrQmwSGFNw9f0ucpR1+KfoBN1HRq UfhzMQyZeojLXdq31IKRLvmOSGkCpprQckMV+g3rfiedRBU0kTQWdHiuidalPNw62U+W xsYFFmktRflFEZWM7fW0rRTf93p+A6Xjw14pV93VfVt5vDEMDa6mrMOszXPG/zFNti6M Odig== X-Gm-Message-State: ALoCoQntjLExAn/QGL08ddh0gYvlTAbbcbqgeyvR4nV8TXDVGvp8rJL2etzQNED5UDcqQJYPi/8L/nqDU+ldGXmCbEEae/qVcA8Ym1ukqZf1UN2u/Pxq4mM3ZSr9RmqITkojPkyQhju+OB49XkxaYfChjdsCm2nbnjy1R/Z+HTCP1i56VxVXC8w= X-Received: by 10.50.154.102 with SMTP id vn6mr66962235igb.1.1388592310066; Wed, 01 Jan 2014 08:05:10 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.50.154.102 with SMTP id vn6mr66962227igb.1.1388592309969; Wed, 01 Jan 2014 08:05:09 -0800 (PST) Received: by 10.64.227.15 with HTTP; Wed, 1 Jan 2014 08:05:09 -0800 (PST) Date: Wed, 1 Jan 2014 18:05:09 +0200 Message-ID: Subject: Setting up Snappy compression in Hadoop From: Amit Sela To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=047d7bd7581e5fe71f04eeead78c X-Virus-Checked: Checked by ClamAV on apache.org --047d7bd7581e5fe71f04eeead78c Content-Type: text/plain; charset=ISO-8859-1 Hi all, I'm running on Hadoop 1.0.4 and I'd like to use Snappy for map output compression. I'm adding the configurations: configuration.setBoolean("mapred.compress.map.output", true); configuration.set("mapred.map.output.compression.codec", "org.apache.hadoop.io.compress.SnappyCodec"); And I've added libsnappy.so.1 to $HADOOP_HOME/lib/native/Linux-amd64-64/ Still, all map tasks fail with "native snappy library not available". Could anyone elaborate on how to install Snappy for Hadoop ? Thanks, Amit. --047d7bd7581e5fe71f04eeead78c Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi all,=A0

I'm running on Hadoop 1.= 0.4 and I'd like to use Snappy for map output compression.
I&= #39;m adding the configurations:

configuration.set= Boolean("mapred.compress.map.output", true);
configuration.set("mapred.map.output.compression.codec"= ;, "org.apache.hadoop.io.compress.SnappyCodec");
And I've added=A0libsnappy.so.1 to $HADOOP_HOME/lib/native= /Linux-amd64-64/

Still, all map tasks fail with "native snappy library not available".
=
Could anyone elaborate on how to install Snappy for Hadoop ?

Thanks,=A0

Amit.
--047d7bd7581e5fe71f04eeead78c--