hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arthur.hk.chan@gmail.com" <arthur.hk.c...@gmail.com>
Subject Re: Compilation error: HBASE 0.98.4 with Snappy
Date Tue, 26 Aug 2014 13:30:07 GMT
Hi JM

Below are my commands, tried two cases under same source code folder: 
a) compile with snappy parameters(failed),  
b) compile without snappy parameters (successful).

Regards
Arthur

wget http://mirrors.devlib.org/apache/hbase/stable/hbase-0.98.4-src.tar.gz
tar -vxf hbase-0.98.4-src.tar.gz
mv hbase-0.98.4 hbase-0.98.4-src_snappy
cd  hbase-0.98.4-src_snappy
nano dev-support/generate-hadoopX-poms.sh
  (change  hbase_home=“/usr/local/hadoop/hbase-0.98.4-src_snappy”)


bash -x ./dev-support/generate-hadoopX-poms.sh 0.98.4 0.98.4-hadoop2
a) with snappy parameters
mvn -f pom.xml.hadoop2 install -DskipTests assembly:single -Prelease,hadoop-snappy -Dhadoop-snappy.version=0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] Building HBase - Server 0.98.4-hadoop2
[INFO] ------------------------------------------------------------------------
[WARNING] The POM for org.apache.hadoop:hadoop-snappy:jar:0.0.1-SNAPSHOT is missing, no dependency
information available
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] HBase ............................................. SUCCESS [8.192s]
[INFO] HBase - Common .................................... SUCCESS [5.638s]
[INFO] HBase - Protocol .................................. SUCCESS [1.535s]
[INFO] HBase - Client .................................... SUCCESS [1.206s]
[INFO] HBase - Hadoop Compatibility ...................... SUCCESS [0.193s]
[INFO] HBase - Hadoop Two Compatibility .................. SUCCESS [0.798s]
[INFO] HBase - Prefix Tree ............................... SUCCESS [0.438s]
[INFO] HBase - Server .................................... FAILURE [0.234s]
[INFO] HBase - Testing Util .............................. SKIPPED
[INFO] HBase - Thrift .................................... SKIPPED
[INFO] HBase - Shell ..................................... SKIPPED
[INFO] HBase - Integration Tests ......................... SKIPPED
[INFO] HBase - Examples .................................. SKIPPED
[INFO] HBase - Assembly .................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 19.474s
[INFO] Finished at: Tue Aug 26 21:21:13 HKT 2014
[INFO] Final Memory: 51M/1100M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project hbase-server: Could not resolve dependencies for
project org.apache.hbase:hbase-server:jar:0.98.4-hadoop2: Failure to find org.apache.hadoop:hadoop-snappy:jar:0.0.1-SNAPSHOT
in http://maven.oschina.net/content/groups/public/ was cached in the local repository, resolution
will not be reattempted until the update interval of nexus-osc has elapsed or updates are
forced -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following
articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hbase-server




b) try again, without snappy parameters
mvn -f pom.xml.hadoop2 install -DskipTests assembly:single -Prelease
[INFO] Building tar: /edh/hadoop_all_sources/hbase-0.98.4-src_snappy/hbase-assembly/target/hbase-0.98.4-hadoop2-bin.tar.gz
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] HBase ............................................. SUCCESS [3.290s]
[INFO] HBase - Common .................................... SUCCESS [3.119s]
[INFO] HBase - Protocol .................................. SUCCESS [0.972s]
[INFO] HBase - Client .................................... SUCCESS [0.920s]
[INFO] HBase - Hadoop Compatibility ...................... SUCCESS [0.167s]
[INFO] HBase - Hadoop Two Compatibility .................. SUCCESS [0.504s]
[INFO] HBase - Prefix Tree ............................... SUCCESS [0.382s]
[INFO] HBase - Server .................................... SUCCESS [4.790s]
[INFO] HBase - Testing Util .............................. SUCCESS [0.598s]
[INFO] HBase - Thrift .................................... SUCCESS [1.536s]
[INFO] HBase - Shell ..................................... SUCCESS [0.369s]
[INFO] HBase - Integration Tests ......................... SUCCESS [0.443s]
[INFO] HBase - Examples .................................. SUCCESS [0.459s]
[INFO] HBase - Assembly .................................. SUCCESS [13.240s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 31.408s
[INFO] Finished at: Tue Aug 26 21:22:50 HKT 2014
[INFO] Final Memory: 57M/1627M
[INFO] ------------------------------------------------------------------------





On 26 Aug, 2014, at 8:52 pm, Jean-Marc Spaggiari <jean-marc@spaggiari.org> wrote:

> Hi Arthur,
> 
> How have you extracted HBase source and what command do you run to build? I
> will do the same here locally so I can provide you the exact step to
> complete.
> 
> JM
> 
> 
> 2014-08-26 8:42 GMT-04:00 Arthur.hk.chan@gmail.com <arthur.hk.chan@gmail.com
>> :
> 
>> Hi JM
>> 
>> Not too sure what you mean, do you mean I should create a new folder in my
>> HBASE_SRC named lib/native/Linux-x86 and copy these files to this folder
>> then try to compile it again?
>> 
>> Regards
>> ARthur
>> 
>> 
>> On 26 Aug, 2014, at 8:17 pm, Jean-Marc Spaggiari <jean-marc@spaggiari.org>
>> wrote:
>> 
>>> Hi Arthur,
>>> 
>>> Almost done! You now need to copy them on the HBase folder.
>>> 
>>> hbase@hbasetest1:~/hbase-0.98.2-hadoop2/lib$ tree | grep -v .jar | grep
>> -v
>>> .rb
>>> .
>>> ├── native
>>> │   └── Linux-x86
>>> │       ├── libsnappy.a
>>> │       ├── libsnappy.la
>>> │       ├── libsnappy.so
>>> │       ├── libsnappy.so.1
>>> │       └── libsnappy.so.1.2.0
>>> 
>>> I don't have any hadoop-snappy lib in my hbase folder and it works very
>>> well with Snappy for me...
>>> 
>>> JM
>>> 
>>> 2014-08-26 8:09 GMT-04:00 Arthur.hk.chan@gmail.com <
>> arthur.hk.chan@gmail.com
>>>> :
>>> 
>>>> Hi JM,
>>>> 
>>>> Below are my steps to install snappy lib, do I miss something?
>>>> 
>>>> Regards
>>>> Arthur
>>>> 
>>>> wget https://snappy.googlecode.com/files/snappy-1.1.1.tar.gz
>>>> tar -vxf snappy-1.1.1.tar.gz
>>>> cd snappy-1.1.1
>>>> ./configure
>>>> make
>>>> make install
>>>>       make[1]: Entering directory
>> `/edh/hadoop_all_sources/snappy-1.1.1'
>>>>       test -z "/usr/local/lib" || /bin/mkdir -p "/usr/local/lib"
>>>>        /bin/sh ./libtool   --mode=install /usr/bin/install -c
>>>> libsnappy.la '/usr/local/lib'
>>>>       libtool: install: /usr/bin/install -c .libs/libsnappy.so.1.2.0
>>>> /usr/local/lib/libsnappy.so.1.2.0
>>>>       libtool: install: (cd /usr/local/lib && { ln -s -f
>>>> libsnappy.so.1.2.0 libsnappy.so.1 || { rm -f libsnappy.so.1 && ln
-s
>>>> libsnappy.so.1.2.0 libsnappy.so.1; }; })
>>>>       libtool: install: (cd /usr/local/lib && { ln -s -f
>>>> libsnappy.so.1.2.0 libsnappy.so || { rm -f libsnappy.so && ln -s
>>>> libsnappy.so.1.2.0 libsnappy.so; }; })
>>>>       libtool: install: /usr/bin/install -c .libs/libsnappy.lai
>>>> /usr/local/lib/libsnappy.la
>>>>       libtool: install: /usr/bin/install -c .libs/libsnappy.a
>>>> /usr/local/lib/libsnappy.a
>>>>       libtool: install: chmod 644 /usr/local/lib/libsnappy.a
>>>>       libtool: install: ranlib /usr/local/lib/libsnappy.a
>>>>       libtool: finish:
>>>> 
>> PATH="/edh/hadoop/spark/bin:/edh/hadoop/hbase/bin:/edh/hadoop/zookeeper//bin:/edh/hadoop/yarn/hadoop/bin:/edh/hadoop/yarn/hadoop/sbin:/usr/lib64/qt-3.3/bin:/opt/apache-maven-3.1.1/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/edh/hadoop/zookeeper//bin:/edh/hadoop/hive//bin:/usr/lib/jvm/jdk1.6.0_45//bin:/root/bin:/sbin"
>>>> ldconfig -n /usr/local/lib
>>>> 
>>>> ----------------------------------------------------------------------
>>>>       Libraries have been installed in:
>>>>       /usr/local/lib
>>>>       If you ever happen to want to link against installed libraries
>>>>       in a given directory, LIBDIR, you must either use libtool, and
>>>>       specify the full pathname of the library, or use the `-LLIBDIR'
>>>>       flag during linking and do at least one of the following:
>>>>       - add LIBDIR to the `LD_LIBRARY_PATH' environment variable
>>>>       during execution
>>>>       - add LIBDIR to the `LD_RUN_PATH' environment variable
>>>>       during linking
>>>>       - use the `-Wl,-rpath -Wl,LIBDIR' linker flag
>>>>       - have your system administrator add LIBDIR to `/etc/ld.so.conf'
>>>>       See any operating system documentation about shared libraries for
>>>>       more information, such as the ld(1) and ld.so(8) manual pages.
>>>> 
>>>> ----------------------------------------------------------------------
>>>>       test -z "/usr/local/share/doc/snappy" || /bin/mkdir -p
>>>> "/usr/local/share/doc/snappy"
>>>>        /usr/bin/install -c -m 644 ChangeLog COPYING INSTALL NEWS README
>>>> format_description.txt framing_format.txt '/usr/local/share/doc/snappy'
>>>>       test -z "/usr/local/include" || /bin/mkdir -p
>> "/usr/local/include"
>>>>        /usr/bin/install -c -m 644 snappy.h snappy-sinksource.h
>>>> snappy-stubs-public.h snappy-c.h '/usr/local/include'
>>>>       make[1]: Leaving directory `/edh/hadoop_all_sources/snappy-1.1.1'
>>>> 
>>>> ll /usr/local/lib
>>>>       -rw-r--r--. 1 root root   233554 Aug 20 00:14 libsnappy.a
>>>>       -rwxr-xr-x. 1 root root      953 Aug 20 00:14 libsnappy.la
>>>>       lrwxrwxrwx. 1 root root       18 Aug 20 00:14 libsnappy.so ->
>>>> libsnappy.so.1.2.0
>>>>       lrwxrwxrwx. 1 root root       18 Aug 20 00:14 libsnappy.so.1 ->
>>>> libsnappy.so.1.2.0
>>>>       -rwxr-xr-x. 1 root root   147726 Aug 20 00:14 libsnappy.so.1.2.0
>>>> 
>>>> 
>>>> 
>>>> On 26 Aug, 2014, at 7:38 pm, Jean-Marc Spaggiari <
>> jean-marc@spaggiari.org>
>>>> wrote:
>>>> 
>>>>> Hi Arthur,
>>>>> 
>>>>> Do you have snappy libs installed and configured? HBase doesn't come
>> with
>>>>> Snappy. So yo need to have it first.
>>>>> 
>>>>> Shameless plug:
>>>>> 
>>>> 
>> http://www.spaggiari.org/index.php/hbase/how-to-install-snappy-with-1#.U_xxSqdZuZY
>>>>> 
>>>>> This is for 0.96 but should be very similar for 0.98. I will try it
>> soon
>>>>> and post and update, but keep us posted here so we can support you...
>>>>> 
>>>>> JM
>>>>> 
>>>>> 
>>>>> 2014-08-26 7:34 GMT-04:00 Arthur.hk.chan@gmail.com <
>>>> arthur.hk.chan@gmail.com
>>>>>> :
>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> I need to install snappy to HBase 0.98.4.  (my Hadoop version is
>> 2.4.1)
>>>>>> 
>>>>>> Can you please advise what would be wrong?  Should my pom.xml be
>>>> incorrect
>>>>>> and missing something?
>>>>>> 
>>>>>> Regards
>>>>>> Arthur
>>>>>> 
>>>>>> 
>>>>>> Below are my commands:
>>>>>> bash -x ./dev-support/generate-hadoopX-poms.sh 0.98.4 0.98.4-hadoop2
>>>>>> mvn -f pom.xml.hadoop2 install -DskipTests assembly:single
>>>>>> -Prelease,hadoop-snappy
>>>>>> 
>>>>>> Iog:
>>>>>> [INFO]
>>>>>> 
>> ------------------------------------------------------------------------
>>>>>> [INFO] Building HBase - Server 0.98.4-hadoop2
>>>>>> [INFO]
>>>>>> 
>> ------------------------------------------------------------------------
>>>>>> [WARNING] The POM for
>> org.apache.hadoop:hadoop-snappy:jar:0.0.1-SNAPSHOT
>>>>>> is missing, no dependency information available
>>>>>> [INFO]
>>>>>> 
>> ------------------------------------------------------------------------
>>>>>> [INFO] Reactor Summary:
>>>>>> [INFO]
>>>>>> [INFO] HBase ............................................. SUCCESS
>>>> [3.129s]
>>>>>> [INFO] HBase - Common .................................... SUCCESS
>>>> [3.105s]
>>>>>> [INFO] HBase - Protocol .................................. SUCCESS
>>>> [0.976s]
>>>>>> [INFO] HBase - Client .................................... SUCCESS
>>>> [0.925s]
>>>>>> [INFO] HBase - Hadoop Compatibility ...................... SUCCESS
>>>> [0.183s]
>>>>>> [INFO] HBase - Hadoop Two Compatibility .................. SUCCESS
>>>> [0.497s]
>>>>>> [INFO] HBase - Prefix Tree ............................... SUCCESS
>>>> [0.407s]
>>>>>> [INFO] HBase - Server .................................... FAILURE
>>>> [0.103s]
>>>>>> [INFO] HBase - Testing Util .............................. SKIPPED
>>>>>> [INFO] HBase - Thrift .................................... SKIPPED
>>>>>> [INFO] HBase - Shell ..................................... SKIPPED
>>>>>> [INFO] HBase - Integration Tests ......................... SKIPPED
>>>>>> [INFO] HBase - Examples .................................. SKIPPED
>>>>>> [INFO] HBase - Assembly .................................. SKIPPED
>>>>>> [INFO]
>>>>>> 
>> ------------------------------------------------------------------------
>>>>>> [INFO] BUILD FAILURE
>>>>>> [INFO]
>>>>>> 
>> ------------------------------------------------------------------------
>>>>>> [INFO] Total time: 9.939s
>>>>>> [INFO] Finished at: Tue Aug 26 19:23:14 HKT 2014
>>>>>> [INFO] Final Memory: 61M/2921M
>>>>>> [INFO]
>>>>>> 
>> ------------------------------------------------------------------------
>>>>>> [ERROR] Failed to execute goal on project hbase-server: Could not
>>>> resolve
>>>>>> dependencies for project
>>>> org.apache.hbase:hbase-server:jar:0.98.4-hadoop2:
>>>>>> Failure to find org.apache.hadoop:hadoop-snappy:jar:0.0.1-SNAPSHOT
in
>>>>>> http://maven.oschina.net/content/groups/public/ was cached in the
>> local
>>>>>> repository, resolution will not be reattempted until the update
>>>> interval of
>>>>>> nexus-osc has elapsed or updates are forced -> [Help 1]
>>>>>> [ERROR]
>>>>>> [ERROR] To see the full stack trace of the errors, re-run Maven with
>> the
>>>>>> -e switch.
>>>>>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>>>>>> [ERROR]
>>>>>> [ERROR] For more information about the errors and possible solutions,
>>>>>> please read the following articles:
>>>>>> [ERROR] [Help 1]
>>>>>> 
>>>> 
>> http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
>>>>>> [ERROR]
>>>>>> [ERROR] After correcting the problems, you can resume the build with
>> the
>>>>>> command
>>>>>> [ERROR]   mvn <goals> -rf :hbase-server
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>> 
>> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message