hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "UsingLzoCompression" by PaulKist
Date Fri, 02 Apr 2010 18:59:18 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "UsingLzoCompression" page has been changed by PaulKist.
http://wiki.apache.org/hadoop/UsingLzoCompression?action=diff&rev1=19&rev2=20

--------------------------------------------------

   * For 0.21 or 0.22 checkout trunk
  
  On Mac:
-  * On Mac you might want to use a command line like:
+  * To install the hadoop-gpl-compression library on a mac, it is advisable to use MacPorts.
To do so you must do the following:
+ (Parts of this found on http://code.google.com/p/hadoop-gpl-compression/wiki/FAQ )
+ 
+ {{{
+ > port fetch lzo2 # If for some reason LZO2 is already installed, please uninstall first
before doing this
+ > port edit lzo2 # A vim editor should open
+ 
+ // Add the following block of text in the file and save the file. 
+ variant x86_64 description "Build the 64-bit." {
+     configure.args-delete     --build=x86-apple-darwin ABI=standard
+     configure.cflags-delete   -m32
+     configure.cxxflags-delete -m32
+ 
+     configure.args-append     --build=x86_64-apple-darwin ABI=64
+     configure.cflags-append   -m64 -arch x86_64
+     configure.cxxflags-append -m64 -arch x86_64
+ }
+ 
+ ## END ##
+ 
+ > port install lzo2 +x86_64
+ }}}
+ 
+ This ensures the library is built in 64 bit mode, because java 1.6 is 64 bit only.  Also
to make sure your lzo library is x64_64 as well, type: {{{
+ $ file /usr/local/lib/liblzo2.2.0.0.dylib
+ /usr/local/lib/liblzo2.2.0.0.dylib: Mach-O 64-bit dynamically linked shared library x86_64
+ }}}
+ 
+ 
+  * On Mac you might want to use a command line, in the hadoop-gpl-compression home directory:
  {{{
  env JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/ \
  C_INCLUDE_PATH=/path/to/lzo64/include LIBRARY_PATH=/path/to/lzo64/lib \
  CFLAGS="-arch x86_64" ant clean compile-native test tar
  }}}
- This ensures the library is built in 64 bit mode, because java 1.6 is 64 bit only.  Also
make sure your lzo library is x64_64 as well, type: {{{
- $ file /usr/local/lib/liblzo2.2.0.0.dylib
- /usr/local/lib/liblzo2.2.0.0.dylib: Mach-O 64-bit dynamically linked shared library x86_64
+ 
+ * Note: If you used macports, /path/to/lzo64 will be replaced by /opt/local  (e.g. /opt/local/include
and /opt/local/lib )
+ * Note: If for some reason you are getting compilation errors, you can add the following
to the environment variables: 
+ 
+ {{{
+ CLASSPATH=$HADOOP_HOME/hadoop-<version>-core.jar
  }}}
+ 
+ * Note: Also during this install, if you are running into permission denied errors, even
as ROOT, you can go ahead and change permissions of those files in order for the build to
complete
+ 
+ Once the install has completed, a jar file and lib files have been created in the HADOOP-GPL-HOME/build
directory.  All these files MUST be copied both into your HADOOP_HOME and HBASE_HOME directories
using the following commands from the HADOOP-GPL-HOME directory:
+ 
+ {{{
+ > cp build/hadoop-gpl-compression-0.1.0-dev/hadoop-gpl-compression-0.1.0-dev.jar $HADOOP_HOME/lib/
+ > cp build/hadoop-gpl-compression-0.1.0-dev/hadoop-gpl-compression-0.1.0-dev.jar $HBASE_HOME/lib/
+ > tar -cBf - -C build/hadoop-gpl-compression-0.1.0-dev/lib/native . | tar -xBvf - -C
$HADOOP_HOME/lib/native
+ > tar -cBf - -C build/hadoop-gpl-compression-0.1.0-dev/lib/native . | tar -xBvf - -C
$HBASE_HOME/lib/native
+ }}}
+ 
+ 
+ 
  To build lzo2 from source in 64 bit mode: {{{
  $ CFLAGS="-arch x86_64" ./configure --build=x86_64-darwin --enable-shared --disable-asm
  <configure output>

Mime
View raw message