accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geoffry Roberts <>
Subject Re: Accumulo and OSGi
Date Mon, 07 Apr 2014 14:07:30 GMT
Thanks Corey,

I appreciate your insights wrt the internal nature of the Hadoop Client.  I
hadn't yet opened it up.  Ideally, we should be able to use these things as
black boxes, or so the theory goes. Nevertheless, I'll look into your notes.

My original question remains: Is the Accumulo Client dependent on the
Hadoop Client fully?  This determination can be made through trial and
error.  But I'm looking to leverage OPE (other people's experience) if it

In the same spirit, does anyone know if all the following are required to
run an Accumulo Client? core, fate, start, trace?  If I attempt to OSGify,
I'm trying to figure how much trouble am I getting into.

Thanks all.

On Sun, Apr 6, 2014 at 11:54 AM, Corey Nolet <> wrote:

> Geoffrey,
> My quick answer is that I needed to adjust my container (Karaf in my case)
> to export the JAAS packages because they come in the JRE. Then I needed to
> make the hadoop bundle import them.
> Also before I forget, Hadoop packages its default xml configurations
> (core-site.xml, core-default.xml, etc...) in the default namespace so they
> can't be exported. For this reason, you need to make sure they are included
> at the root bundle level of any bundles needing to use the client (this
> should pertain to the libraries needing the filesystem object, not bundles
> that just  need the Text object, for instance...).
> That's my quick answer on my phone. Ill look into the rest and provide a
> more detailed description of what I did.
> I went through this headache too and it would help to capture the solution
> somewhere (like this thread)
> On Apr 6, 2014 11:18 AM, "Geoffry Roberts" <> wrote:
>> All,
>> To what extent does the Accumulo Client rely on the Hadoop Client?  I
>> apologize if the question is a bit obtuse.  But I got into dependency weeds
>> trying to get the Hadoop Client to work in OSGI.  (See below Hadoop Client
>> woes)   I am now wondering if I OSGified Accumulo's client would I
>> encounter the same-old-same-old or somehow dodge the bullet.
>> Would anyone else be interested?
>> <Hadoop Client woes>
>> I sat down to do what Corey suggested and took a shot at getting the
>> Hadoop Client working in OSGi.  I used the service mix bundles, but alas,
>> it seems that somewhere in the dependencies something wants to use JAAS and
>> that is stopping the show.  I creating a fragment that exports the required
>> package so OSGI can find them--nothing doing; I'm stuck.
>> If one Googles, one finds JAAS is problematic in OSGi as are a number of
>> J2EE technologies.
>> </Hadoop Client woes>
>> On Tue, Apr 1, 2014 at 11:22 AM, Geoffry Roberts <>wrote:
>>> Thank you Corey,
>>> I was unaware of the service mix Hadoop client.  It's funny that no one
>>> on the Hadoop list ever mentioned it.
>>> You say you have 1.4 working in OSGi. Did you do a proper port or just
>>> wrap it with something like bnd?   I have Hadoop 2.3.0 so I need to use
>>> Accumulo 1.5.1.  I'm glad to hear the bane of split packages has been put
>>> asunder.
>>> I am using equinox for now.  Most of my legacy code in based on bndtools
>>> and I exercise it in equinox.  I've used equinox in the past for production
>>> and things went well.
>>> I just grabbed the Hadoop core & client from service mix then zookeeper.
>>> At the least, they were accepted into my bndtools repository as valid
>>> bundles.  I noticed that a number of the usual Hadoop dependencies are in
>>> service mix as well so I sense a glimmer of hope. Wrt Accumulo, I think
>>> I'll take a stab at either wrapping or porting core, frame, start, and
>>> trace to OSGi and see how much trouble I get into.
>>> It's either do the above or abandon OSGi altogether for this project.
>>> My objective is to persist EMF object graphs into Accumulo.  These
>>> graphs were built by others and are based on ISO standards so I need to
>>> walk the straight and narrow and not drop a stitch.  I have code that does
>>> what I need (persist any graph sight unseen) into MongoDB.  I need to adapt
>>> said code to Accumulo. All the above is OSGi based so it would really help
>>> if I can keep the Accumulo end of things in OSGi as well.
>>> Wish me well
>>> On Mon, Mar 31, 2014 at 8:34 PM, Corey Nolet <> wrote:
>>>> Geoffry,
>>>> What OSGi container are you using currently? The servicemix Hadoop
>>>> bundle should get you going with the Hadoop client dependencies at least
>>>> [1]. It looks like one of the servicemix guys created a Hadoop ticket for
>>>> making bundles of their jars as well [2], though it doesn't look like
>>>> there's been any movement on it.
>>>> I recently had to get the CDH3u4 client code working in Karaf. A good
>>>> starting place for me was [3], however I did need to make updates to
>>>> versions of many of the dependencies to get it functioning as expected. [3]
>>>> will get you at least started with dependent bundles and the proper
>>>> imports/exports to get it working.
>>>> I've got the Accumulo client running in OSGi. If I recall correctly,
>>>> versions 1.4 and above do not split packages across jars so it's really
>>>> just a matter of getting the dependencies right. Zookeeper also ships as
>>>> bundle [4].
>>>> Hope this helps.
>>>> [1]
>>>>  [2]
>>>> [3]
>>>> [4]
>>>> On Mon, Mar 31, 2014 at 11:37 AM, Geoffry Roberts <
>>>>> wrote:
>>>>> Luk,
>>>>> Thanks for the link, but I am a bit lost.  wso2 offers middleware,
>>>>> apparently you believe this will help my situation.  If it's not too
>>>>> can you expand?
>>>>> On Mon, Mar 31, 2014 at 11:13 AM, Luk Vervenne <>wrote:
>>>>>> osgi... see
>>>>>> On 31 Mar 2014, at 16:58, Geoffry Roberts <>
>>>>>> wrote:
>>>>>> > All,
>>>>>> >
>>>>>> > I have a project for which Accumulo it appears will serve well.
>>>>>>  However, I have a significant amount of code I want to leverage
that runs
>>>>>> in OSGi.  I don't need for Accumulo itself to be OSGi based but the
>>>>>> Accumulo client yes.  I see that the Accumulo client uses all the
>>>>>> dependencies of the Hadoop client and therefore is not OSGi ready
at this
>>>>>> time.  The Hadoop client certainly doesn't do OSGi--I don't think
it can
>>>>>> even spell it :-)--and attempting to make it so starts turning into
a sure
>>>>>> path to a long sojourn through dependency hell.  I know, I've tried.
>>>>>> >
>>>>>> > Nonetheless, I would like to ask: Is there any interest in the
>>>>>> Accumulo world of having an OSGi based client for this otherwise
>>>>>> appealing database?
>>>>>> >
>>>>>> > Thanks mucho
>>>>>> > --
>>>>>> > There are ways and there are ways,
>>>>>> >
>>>>>> > Geoffry Roberts
>>>>> --
>>>>> There are ways and there are ways,
>>>>> Geoffry Roberts
>>> --
>>> There are ways and there are ways,
>>> Geoffry Roberts
>> --
>> There are ways and there are ways,
>> Geoffry Roberts

There are ways and there are ways,

Geoffry Roberts

View raw message