accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geoffry Roberts <threadedb...@gmail.com>
Subject Re: Accumulo and OSGi
Date Sun, 06 Apr 2014 15:01:01 GMT
All,

To what extent does the Accumulo Client rely on the Hadoop Client?  I
apologize if the question is a bit obtuse.  But I got into dependency weeds
trying to get the Hadoop Client to work in OSGI.  (See below Hadoop Client
woes)   I am now wondering if I OSGified Accumulo's client would I
encounter the same-old-same-old or somehow dodge the bullet.

Would anyone else be interested?

<Hadoop Client woes>
I sat down to do what Corey suggested and took a shot at getting the Hadoop
Client working in OSGi.  I used the service mix bundles, but alas, it seems
that somewhere in the dependencies something wants to use JAAS and that is
stopping the show.  I creating a fragment that exports the required package
so OSGI can find them--nothing doing; I'm stuck.

If one Googles, one finds JAAS is problematic in OSGi as are a number of
J2EE technologies.
</Hadoop Client woes>


On Tue, Apr 1, 2014 at 11:22 AM, Geoffry Roberts <threadedblue@gmail.com>wrote:

> Thank you Corey,
>
> I was unaware of the service mix Hadoop client.  It's funny that no one on
> the Hadoop list ever mentioned it.
>
> You say you have 1.4 working in OSGi. Did you do a proper port or just
> wrap it with something like bnd?   I have Hadoop 2.3.0 so I need to use
> Accumulo 1.5.1.  I'm glad to hear the bane of split packages has been put
> asunder.
>
> I am using equinox for now.  Most of my legacy code in based on bndtools
> and I exercise it in equinox.  I've used equinox in the past for production
> and things went well.
>
> I just grabbed the Hadoop core & client from service mix then zookeeper.
> At the least, they were accepted into my bndtools repository as valid
> bundles.  I noticed that a number of the usual Hadoop dependencies are in
> service mix as well so I sense a glimmer of hope. Wrt Accumulo, I think
> I'll take a stab at either wrapping or porting core, frame, start, and
> trace to OSGi and see how much trouble I get into.
>
> It's either do the above or abandon OSGi altogether for this project.
>
> My objective is to persist EMF object graphs into Accumulo.  These graphs
> were built by others and are based on ISO standards so I need to walk the
> straight and narrow and not drop a stitch.  I have code that does what I
> need (persist any graph sight unseen) into MongoDB.  I need to adapt said
> code to Accumulo. All the above is OSGi based so it would really help if I
> can keep the Accumulo end of things in OSGi as well.
>
> Wish me well
>
>
> On Mon, Mar 31, 2014 at 8:34 PM, Corey Nolet <cjnolet@gmail.com> wrote:
>
>> Geoffry,
>>
>> What OSGi container are you using currently? The servicemix Hadoop bundle
>> should get you going with the Hadoop client dependencies at least [1]. It
>> looks like one of the servicemix guys created a Hadoop ticket for making
>> bundles of their jars as well [2], though it doesn't look like there's been
>> any movement on it.
>>
>> I recently had to get the CDH3u4 client code working in Karaf. A good
>> starting place for me was [3], however I did need to make updates to
>> versions of many of the dependencies to get it functioning as expected. [3]
>> will get you at least started with dependent bundles and the proper
>> imports/exports to get it working.
>>
>> I've got the Accumulo client running in OSGi. If I recall correctly,
>> versions 1.4 and above do not split packages across jars so it's really
>> just a matter of getting the dependencies right. Zookeeper also ships as a
>> bundle [4].
>>
>>
>> Hope this helps.
>>
>> [1]
>> http://mvnrepository.com/artifact/org.apache.servicemix.bundles/org.apache.servicemix.bundles.hadoop-core/
>>  [2] https://issues.apache.org/jira/browse/HADOOP-8446
>> [3] https://github.com/jbonofre/karaf-hadoop
>> [4] https://issues.apache.org/jira/browse/ZOOKEEPER-425
>>
>>
>>
>> On Mon, Mar 31, 2014 at 11:37 AM, Geoffry Roberts <threadedblue@gmail.com
>> > wrote:
>>
>>> Luk,
>>>
>>> Thanks for the link, but I am a bit lost.  wso2 offers middleware,
>>> apparently you believe this will help my situation.  If it's not too much,
>>> can you expand?
>>>
>>>
>>> On Mon, Mar 31, 2014 at 11:13 AM, Luk Vervenne <luk@synergetics.be>wrote:
>>>
>>>> osgi... see wso2.com
>>>>
>>>> On 31 Mar 2014, at 16:58, Geoffry Roberts <threadedblue@gmail.com>
>>>> wrote:
>>>>
>>>> > All,
>>>> >
>>>> > I have a project for which Accumulo it appears will serve well.
>>>>  However, I have a significant amount of code I want to leverage that runs
>>>> in OSGi.  I don't need for Accumulo itself to be OSGi based but the
>>>> Accumulo client yes.  I see that the Accumulo client uses all the
>>>> dependencies of the Hadoop client and therefore is not OSGi ready at this
>>>> time.  The Hadoop client certainly doesn't do OSGi--I don't think it can
>>>> even spell it :-)--and attempting to make it so starts turning into a sure
>>>> path to a long sojourn through dependency hell.  I know, I've tried.
>>>> >
>>>> > Nonetheless, I would like to ask: Is there any interest in the
>>>> Accumulo world of having an OSGi based client for this otherwise very
>>>> appealing database?
>>>> >
>>>> > Thanks mucho
>>>> > --
>>>> > There are ways and there are ways,
>>>> >
>>>> > Geoffry Roberts
>>>>
>>>>
>>>
>>>
>>> --
>>> There are ways and there are ways,
>>>
>>> Geoffry Roberts
>>>
>>
>>
>
>
> --
> There are ways and there are ways,
>
> Geoffry Roberts
>



-- 
There are ways and there are ways,

Geoffry Roberts

Mime
View raw message