hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Viral Bajaria <viral.baja...@gmail.com>
Subject Re: hbase 0.94.4 with hadoop 0.23.5
Date Tue, 29 Jan 2013 02:26:32 GMT
Just closing the loop here, it might help someone else to hand patch their
build process before I get the patches in the hadoop branch, no changes
required for hbase.

I backported the latest version of KerberosUtil from hadoop 2.0 branch and
recompiled hadoop-common/hadoop-auth and then installed the jar out to my
local maven repository. Ran the command "mvn clean package
-Dhadoop.profile=23" and I was able to build hbase against hadoop 0.23. Now
starts the more painful part of making sure everything works during runtime
:-)

Stack, I noticed that in all profiles except 0.23 there is hadoop-core or
hadoop-common includes, while in 0.23 there is only hadoop-client as a
dependency and there is no mention for hadoop-common or hadoop-auth
anywhere, do they get pulled in due to other dependencies ? Just trying to
understand the whole build process for hbase.

Thanks,
Viral

On Mon, Jan 28, 2013 at 5:58 PM, Viral Bajaria <viral.bajaria@gmail.com>wrote:

> Tried all of it, I think I will have to defer this to the hadoop mailing
> list because it seems there is a missing class in hadoop 0.23 branches, not
> sure if that is intentional. The class exists in trunk and hadoop 2.0
> branches. Though the surprising part is that it does not exist in < 0.23.
> Does the hbase code base exclude certain files based on what hadoop profile
> we are targeting ?
>
> I will try to apply a patch of that class to 0.23.5 and see if it works
> and if it does then I will post it to the list and submit a patch to both
> hadoop and hbase (needs pom change).
>
> -Viral
>
>
> On Mon, Jan 28, 2013 at 5:34 PM, Stack <stack@duboce.net> wrote:
>
>> The below seems like a good suggestion by Vandana.
>>
>> I will say that focus is on support for hadoop 1 and 2.   There has not
>> been much call for us to support 0.23.x  If you can figure what needs
>> fixing, we could try adding the fix to 0.94 (In trunk a patch to add a
>> compatibility module for hadoop-0.23.x would be welcomed).
>>
>> St.Ack
>>
>>
>> On Mon, Jan 28, 2013 at 5:09 PM, Vandana Ayyalasomayajula <
>> avandana@yahoo-inc.com> wrote:
>>
>> > May be thats the issue. Try downloading the source from 0.94 branch and
>> > use the maven command with -Psecurity and -Dhadoop.profile=23.
>> > That should work.
>> >
>> > Thanks
>> > Vandana
>> >
>> > On Jan 28, 2013, at 11:48 AM, Viral Bajaria wrote:
>> >
>> > > Thanks Vandana for reply. I tried that but no luck. It still throws
>> the
>> > > same error. I thought there might have been a typo and you meant -D
>> and
>> > not
>> > > -P but none of them worked.
>> > >
>> > > I verified that the hadoop-auth code base does not have KerberosUtil
>> > class
>> > > anymore. So I am guessing there is some, but I am surprised no one has
>> > > raised this point till now because on the list a few of them say that
>> > they
>> > > are using hbase 0.94 with hadoop 0.23
>> > >
>> > > Maybe I am doing something totally wrong and downloading via the
>> > > "downloads" link and compiling is not the right thing to do and I
>> should
>> > > just get it from the source repository ?
>> > >
>> > > -Viral
>> > >
>> > > On Mon, Jan 28, 2013 at 7:43 AM, Vandana Ayyalasomayajula <
>> > > avandana@yahoo-inc.com> wrote:
>> > >
>> > >> Hi viral,
>> > >>
>> > >> Try adding "-Psecurity" and then compiling.
>> > >>
>> > >> Thanks
>> > >> Vandana
>> > >>
>> > >> Sent from my iPhone
>> > >>
>> > >> On Jan 28, 2013, at 3:05 AM, "Viral Bajaria" <
>> viral.bajaria@gmail.com>
>> > >> wrote:
>> > >>
>> > >>> Hi,
>> > >>>
>> > >>> Is anyone running hbase 0.94.4 against hadoop 0.23.5 ? If yes,
how
>> did
>> > >> you
>> > >>> end up compiling hbase for hadoop 0.23 ?
>> > >>>
>> > >>> I downloaded the hbase release and tried running "mvn clean package
>> > >>> -Dhadoop.profile=23" but I keep on getting a compilation error
as
>> > >> follows:
>> > >>>
>> > >>> [INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile)
@
>> > hbase
>> > >> ---
>> > >>> [INFO] Compiling 738 source files to
>> > >>> /home/viral/dev/downloads/hbase-0.94.4/target/classes
>> > >>> [INFO] -------------------------------------------------------------
>> > >>> [ERROR] COMPILATION ERROR :
>> > >>> [INFO] -------------------------------------------------------------
>> > >>> [ERROR]
>> > >>>
>> > >>
>> >
>> /home/viral/dev/downloads/hbase-0.94.4/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java:[41,53]
>> > >>> cannot find symbol
>> > >>> symbol  : class KerberosUtil
>> > >>> location: package org.apache.hadoop.security.authentication.util
>> > >>> [ERROR]
>> > >>>
>> > >>
>> >
>> /home/viral/dev/downloads/hbase-0.94.4/src/main/java/org/apache/hadoop/hbase/util/Bytes.java:[45,15]
>> > >>> sun.misc.Unsafe is Sun proprietary API and may be removed in a
>> future
>> > >>> release
>> > >>> [ERROR]
>> > >>>
>> > >>
>> >
>> /home/viral/dev/downloads/hbase-0.94.4/src/main/java/org/apache/hadoop/hbase/util/Bytes.java:[1029,19]
>> > >>> sun.misc.Unsafe is Sun proprietary API and may be removed in a
>> future
>> > >>> release
>> > >>> [ERROR]
>> > >>>
>> > >>
>> >
>> /home/viral/dev/downloads/hbase-0.94.4/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java:[242,32]
>> > >>> cannot find symbol
>> > >>> symbol  : variable KerberosUtil
>> > >>> location: class
>> > >> org.apache.hadoop.hbase.zookeeper.ZKUtil.JaasConfiguration
>> > >>>
>> > >>> Tried with both the security release and the one without but no
>> luck.
>> > >>>
>> > >>> Any pointers ?
>> > >>>
>> > >>> Thanks,
>> > >>> Viral
>> > >>
>> >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message