Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Tue, 20 Dec 2011 23:41:31 +0000 (UTC)
From: "Wing Yew Poon (Resolved) (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: 
 <1211065919.33106.1324424491967.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1306857706.47353.1323211719914.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Resolved] (MAPREDUCE-3515) hadoop 0.23: native compression
 libraries not being loaded
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/MAPREDUCE-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wing Yew Poon resolved MAPREDUCE-3515.
--------------------------------------

    Resolution: Duplicate
    
> hadoop 0.23: native compression libraries not being loaded
> ----------------------------------------------------------
>
>                 Key: MAPREDUCE-3515
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3515
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Wing Yew Poon
>
> I installed the hadoop package from the Bigtop hadoop 0.23 branch. Among other files, the package installs
> /usr/lib/hadoop/lib/native
> /usr/lib/hadoop/lib/native/libhadoop.a
> /usr/lib/hadoop/lib/native/libhadoop.so.1
> /usr/lib/hadoop/lib/native/libhadoop.so.1.0.0
> /usr/lib/hadoop/lib/native/libhdfs.a
> I ran a simple job using compression:
> hadoop jar /usr/lib/hadoop/hadoop-mapreduce-examples.jar wordcount -D mapreduce.output.fileoutputformat.compress=true -D mapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.GzipCodec examples/text wordcount-gz
> I see
> 11/12/06 13:42:06 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> 11/12/06 13:42:06 WARN snappy.LoadSnappy: Snappy native library not loaded
> and at the end, I see
> -rw-r--r--   1 root supergroup          0 2011-12-06 13:42 wordcount-gz/_SUCCESS
> -rw-r--r--   1 root supergroup      46228 2011-12-06 13:42 wordcount-gz/part-r-00000.gz
> so the output is compressed, but from the log message, I assume that the native library is not being loaded and that the java gzip is being used.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira