Mailing-List: contact hadoop-commits-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
From: Apache Wiki <wikidiffs@apache.org>
To: hadoop-commits@lucene.apache.org
Date: Sat, 13 Jan 2007 14:27:24 -0000
Message-ID: <20070113142724.14976.89395@eos.apache.osuosl.org>
Subject: [Lucene-hadoop Wiki] Update of "NativeHadoop" by Arun C Murthy

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by Arun C Murthy:
http://wiki.apache.org/lucene-hadoop/NativeHadoop

------------------------------------------------------------------------------
- == Overview ==
+ ==== Overview ====
  
  Hadoop has native implementations of certain components for reasons of both performace & non-availability of java-implementations. 
  This section describes the usage & details on how to build the native-libraries.
  
- == Native Hadoop Libraries ==
+ ==== Native Hadoop Libraries ====
  
- Hadoop has the following native-libraries:
+ Hadoop has the following native components:
-    * [http://www.zlib.net zlib] compression codec - Reworked zlib codec using nio's direct-buffers which gives us 60%-70% speedup. (more details [http://issues.apache.org/jira/browse/HADOOP-538 here]
-    * [http://www.oberhumer.com/opensource/lzo/ lzo] compression codec - Implemented due to lack of java bindings for lzo . (more details [http://issues.apache.org/jira/browse/HADOOP-851 here])
+    * [http://www.zlib.net zlib] [http://lucene.apache.org/hadoop/api/org/apache/hadoop/io/compress/CompressionCodec.html compression codec] - Reworked zlib codec using nio's direct-buffers which gives us 60%-70% speedup. (more details [http://issues.apache.org/jira/browse/HADOOP-538 here])
+    * [http://www.oberhumer.com/opensource/lzo/ lzo] [http://lucene.apache.org/hadoop/api/org/apache/hadoop/io/compress/CompressionCodec.html compression codec] - Implemented due to lack of java bindings for lzo . (more details [http://issues.apache.org/jira/browse/HADOOP-851 here])
  
- == Usage ==
+ ==== Usage ====
  
  It is fairly simple to use the native-hadoop libraries:
     * Take a look at the [#Platforms supported platforms]
@@ -30, +30 @@

  [[BR]]
  {{{INFO util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  }}}
- recheck the above steps.
+ If so, recheck the above steps.
  
  [[Anchor(Platforms)]]
- == Supported Platforms ==
+ ==== Supported Platforms ====
  
- Native-hadoop library is supported for '''*nix''' platforms only. Unfortunately it is known not to work on [http://www.cygwin.com Cygwin] and [http://www.apple.com/macosx Mac OS X] and has mainly been used on the Linux platform.
+ Native-hadoop library is supported for '''*nix''' platforms only. Unfortunately it is known ''not to work'' on [http://www.cygwin.com Cygwin] and [http://www.apple.com/macosx Mac OS X] and has mainly been used on the Linux platform. [http://wiki.apache.org/lucene-hadoop/HowToContribute Patches] from anyone interested in getting them working on Cygwin/MacOSX are welcome!
  
  It has been tested on the following Linux distributions:
     * [http://www.redhat.com/rhel/ RHEL4]/[http://fedora.redhat.com/ Fedora]
     * [http://www.ubuntu.com Ubuntu]
     * [http://www.gentoo.org Gentoo]
  
+ On all the above platforms a 32/64 bit native-hadoop library will work with a respective 32/64 bit jvm.
+ 
+ [[Anchor(BuildNativeHadoop)]]
+ ==== Building Native Hadoop Libraries ====
+ 
+ Native-hadoop library is written in [http://en.wikipedia.org/wiki/ANSI_C ANSI C] and built using the GNU autotools-chain (autoconf, autoheader, automake, autoscan, libtool). This means it should be straight-forward to build them on any platform with a standards compliant C compiler and the GNU autotools-chain. (See [#Platforms supported platforms])
+ 
+ In particular the various packages you would need on the target platform are:
+    * C compiler (e.g. [http://gcc.gnu.org/ GNU C Compiler]
+    * GNU Autools Chain
+       * [http://www.gnu.org/software/autoconf/ autoconf]
+       * [http://www.gnu.org/software/automake/ automake]
+       * [http://www.gnu.org/software/libtool/ libtool]
+    * [http://www.zlib.net/ zlib-development package]
+    * [http://www.oberhumer.com/opensource/lzo/ lzo-development package]
+ 
+ '''Note''': It is '''mandatory''' to have both the zlib and lzo development packages on the target platform for ''building'' the native-hadoop library; however for deployment it is sufficient to have install zlib ''or'' lzo if you wish to use only one of them.
+