hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wing Yew Poon (Resolved) (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-3515) hadoop 0.23: native compression libraries not being loaded
Date Tue, 20 Dec 2011 23:41:31 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Wing Yew Poon resolved MAPREDUCE-3515.
--------------------------------------

    Resolution: Duplicate
    
> hadoop 0.23: native compression libraries not being loaded
> ----------------------------------------------------------
>
>                 Key: MAPREDUCE-3515
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3515
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Wing Yew Poon
>
> I installed the hadoop package from the Bigtop hadoop 0.23 branch. Among other files,
the package installs
> /usr/lib/hadoop/lib/native
> /usr/lib/hadoop/lib/native/libhadoop.a
> /usr/lib/hadoop/lib/native/libhadoop.so.1
> /usr/lib/hadoop/lib/native/libhadoop.so.1.0.0
> /usr/lib/hadoop/lib/native/libhdfs.a
> I ran a simple job using compression:
> hadoop jar /usr/lib/hadoop/hadoop-mapreduce-examples.jar wordcount -D mapreduce.output.fileoutputformat.compress=true
-D mapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.GzipCodec
examples/text wordcount-gz
> I see
> 11/12/06 13:42:06 WARN util.NativeCodeLoader: Unable to load native-hadoop library for
your platform... using builtin-java classes where applicable
> 11/12/06 13:42:06 WARN snappy.LoadSnappy: Snappy native library not loaded
> and at the end, I see
> -rw-r--r--   1 root supergroup          0 2011-12-06 13:42 wordcount-gz/_SUCCESS
> -rw-r--r--   1 root supergroup      46228 2011-12-06 13:42 wordcount-gz/part-r-00000.gz
> so the output is compressed, but from the log message, I assume that the native library
is not being loaded and that the java gzip is being used.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message