Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Sat, 15 Mar 2014 01:15:43 +0000 (UTC)
From: "Travis Thompson (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.12701629.1394839707121.74537.1394846143761@arcas>
In-Reply-To: <JIRA.12701629.1394839707121@arcas>
References: <JIRA.12701629.1394839707121@arcas>
Subject: [jira] [Commented] (HADOOP-10409) Bzip2 error message isn't clear
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/HADOOP-10409?page=3Dcom.atlassi=
an.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D13=
935912#comment-13935912 ]=20

Travis Thompson commented on HADOOP-10409:
------------------------------------------

I'd be happy to, I've been meaning to document my build process as it is.

> Bzip2 error message isn't clear
> -------------------------------
>
>                 Key: HADOOP-10409
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10409
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io
>    Affects Versions: 2.3.0
>            Reporter: Travis Thompson
>
> If you compile hadoop without {{bzip2-devel}} installed (on RHEL), bzip2 =
doesn't get compiled into libhadoop, as is expected.  This is not documente=
d however and the error message thrown from {{hadoop checknative -a}} is no=
t helpful.
> {noformat}
> [tthompso@eat1-hcl4060 bin]$ hadoop checknative -a
> 14/03/13 00:51:02 WARN bzip2.Bzip2Factory: Failed to load/initialize nati=
ve-bzip2 library system-native, will use pure-Java version
> 14/03/13 00:51:02 INFO zlib.ZlibFactory: Successfully loaded & initialize=
d native-zlib library
> Native library checking:
> hadoop: true /export/apps/hadoop/hadoop-2.3.0.li7-1-bin/lib/native/libhad=
oop.so.1.0.0
> zlib:   true /lib64/libz.so.1
> snappy: true /usr/lib64/libsnappy.so.1
> lz4:    true revision:99
> bzip2:  false=20
> 14/03/13 00:51:02 INFO util.ExitUtil: Exiting with status 1
> {noformat}
> You can see that it wasn't compiled in here:
> {noformat}
> [mislam@eat1-hcl4060 ~]$ strings /export/apps/hadoop/latest/lib/native/li=
bhadoop.so | grep initIDs
> Java_org_apache_hadoop_io_compress_lz4_Lz4Compressor_initIDs
> Java_org_apache_hadoop_io_compress_lz4_Lz4Decompressor_initIDs
> Java_org_apache_hadoop_io_compress_snappy_SnappyCompressor_initIDs
> Java_org_apache_hadoop_io_compress_snappy_SnappyDecompressor_initIDs
> Java_org_apache_hadoop_io_compress_zlib_ZlibCompressor_initIDs
> Java_org_apache_hadoop_io_compress_zlib_ZlibDecompressor_initIDs
> {noformat}
> After installing bzip2-devel and recompiling:
> {noformat}
> [tthompso@eat1-hcl4060 ~]$ hadoop checknative -a
> 14/03/14 23:00:08 INFO bzip2.Bzip2Factory: Successfully loaded & initiali=
zed native-bzip2 library system-native
> 14/03/14 23:00:08 INFO zlib.ZlibFactory: Successfully loaded & initialize=
d native-zlib library
> Native library checking:
> hadoop: true /export/apps/hadoop/hadoop-2.3.0.11-2-bin/lib/native/libhado=
op.so.1.0.0
> zlib:   true /lib64/libz.so.1
> snappy: true /usr/lib64/libsnappy.so.1
> lz4:    true revision:99
> bzip2:  true /lib64/libbz2.so.1
> {noformat}
> {noformat}
> tthompso@esv4-hcl261:~/hadoop-common(li-2.3.0=E2=9A=A1) =C2=BB strings ./=
hadoop-common-project/hadoop-common/target/native/target/usr/local/lib/libh=
adoop.so |grep initIDs
> Java_org_apache_hadoop_io_compress_lz4_Lz4Compressor_initIDs
> Java_org_apache_hadoop_io_compress_lz4_Lz4Decompressor_initIDs
> Java_org_apache_hadoop_io_compress_snappy_SnappyCompressor_initIDs
> Java_org_apache_hadoop_io_compress_snappy_SnappyDecompressor_initIDs
> Java_org_apache_hadoop_io_compress_zlib_ZlibCompressor_initIDs
> Java_org_apache_hadoop_io_compress_zlib_ZlibDecompressor_initIDs
> Java_org_apache_hadoop_io_compress_bzip2_Bzip2Compressor_initIDs
> Java_org_apache_hadoop_io_compress_bzip2_Bzip2Decompressor_initIDs
> {noformat}
> The error message thrown should hint that perhaps libhadoop wasn't compil=
ed with the bzip2 headers installed.  It would also be nice if compile time=
 dependencies were documented somewhere... :)


--
This message was sent by Atlassian JIRA
(v6.2#6252)