Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4A0811112D for ; Wed, 9 Apr 2014 15:50:10 +0000 (UTC) Received: (qmail 80917 invoked by uid 500); 9 Apr 2014 15:50:09 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 80526 invoked by uid 500); 9 Apr 2014 15:50:07 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 80516 invoked by uid 99); 9 Apr 2014 15:50:05 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 09 Apr 2014 15:50:05 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of threadedblue@gmail.com designates 74.125.82.46 as permitted sender) Received: from [74.125.82.46] (HELO mail-wg0-f46.google.com) (74.125.82.46) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 09 Apr 2014 15:49:59 +0000 Received: by mail-wg0-f46.google.com with SMTP id b13so2643891wgh.17 for ; Wed, 09 Apr 2014 08:49:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=RTdeUH+5cZ0BGX+wFTbUKxZPLJvAMi2vAAYBje9qf00=; b=i4xHUf9Pn+T/2tDM4lno7Vt/P3FWDaGGVX8wfQcGzZZj/fDvF/q0gwisuZHkUvuMhS WtgL7K+E1uzopaWVNlTol6FaBAMw21GYnu5TfAKoPq5lj7ZBfu43PyBZ+975xqhOOyXL V/LSMbDEscxfDNYIx3scbHntaNv3/wMCgXpeyIV1eUSqZINI8ukIZet3LgIsQepX9oH7 MJ1Ekk2ww7KT6CKRe974HTlc1u3HFu026fZUg2imZhsx1/12Kp8gaNiQXUDYx7DutX/R wF0dMvHI/YJbvbIHYvliz5MtKVTMtUy9CfaF6A1u1+Ohg8DGlTRtyE3SnC9+EpHewifI JIxg== MIME-Version: 1.0 X-Received: by 10.180.78.225 with SMTP id e1mr29629869wix.17.1397058578271; Wed, 09 Apr 2014 08:49:38 -0700 (PDT) Received: by 10.194.92.12 with HTTP; Wed, 9 Apr 2014 08:49:38 -0700 (PDT) In-Reply-To: References: <8A1EF275-5034-4C0F-9F9B-251B2307C308@synergetics.be> <5342B5F1.5080108@gmail.com> <5342BDF3.7080203@gmail.com> Date: Wed, 9 Apr 2014 11:49:38 -0400 Message-ID: Subject: Re: Accumulo and OSGi From: Geoffry Roberts To: Accumulo Content-Type: multipart/alternative; boundary=f46d043c086449d30a04f69e0c17 X-Virus-Checked: Checked by ClamAV on apache.org --f46d043c086449d30a04f69e0c17 Content-Type: text/plain; charset=ISO-8859-1 Corey, Interesting you have Hadoop working in Karaf. I'm using equinox. It also sounds as if I don't need to access HFDS in order to get Accumulo to work in OSGi. If I understand you correctly, I only need have Text available. I'll look into that. It does answer my question and maybe I can avoid the JAAS mishmash. I have been using OSGi off and on for a few years now. Close on to 100% of my troubles has always been with getting 3rd party libraries working. Accumulo/Hadoop is only the latest round. I'm gearing to do some major work based on Accumulo. It's security, scaleability, relationship to Hadoop's HDFS and MR all conspire to make it attractive. But creating an uberbundle? I'm sure I could get it working as a proof of concept, but will it play in prime-time? How hard would it be to make a proper Accumulo bundle? Would the community accept it? Thanks So far as my travails with JAAS is concerned, I did this in my bndtools *.bndrun file: -runproperties: org.osgi.framework.system.packages.extra=com.sun.security.auth.module I also tried: Import-Package: com.sun.security.auth.module in my bundle that calls Hadoop. Both of these were advised by one of the bndtools gurus--neither worked. When I did the Import-Package other things broke. On Wed, Apr 9, 2014 at 9:46 AM, Corey Nolet wrote: > Geoffry, > > As Josh pointed out, you should only need the Hadoop libraries on the > client side to use the Text object. This means you won't have to go through > the pain of placing the xml files in your root bundles. > > Did you try the JAAS export from the packages in your container? Did that > help? > > I agree with your comment that we *should* be able to look at these > bundles as black boxes but we're not dealing with true bundles, yet. Once I > got my Hadoop client bundle ready to go, I haven't had to touch it (and the > Hadoop Karaf project solved most of the import/export guessing work for me > already). For Accumulo, on the other hand, it's up to you if you want to > create an uber-bundle with the necessary Accumulo packages or just wrap > each one and provide the necessary imports/exports across them. For my > needs, I just created one uber-bundle for Accumulo packages and imported > the necessary non-Accumulo packages (many of those I had to wrap as well). > I didn't find that part too painful on the Accumulo side. While not being > the ideal situation from the OSGI side, with lack of an existing bundle > artifact, it did solve my problem and I'm actively using both Accumulo and > Hadoop in OSGi. My recommendation would be that until we get a proper > bundle, that solution would certainly work in the short-term. > > I believe Josh posted this already but check out [1]. A ready-to-go OSGi > bundle for Accumulo would be useful but the Hadoop client dependency would > need to be wrapped (or exposed as its own bundle). IMO, with proper > documentation this shouldn't be too painful for users. Thoughts? > > > [1] https://issues.apache.org/jira/browse/ACCUMULO-2518 > > > On Mon, Apr 7, 2014 at 11:27 AM, Geoffry Roberts wrote: > >> Ahh, let me try and address where I might have gone off the linguistic >> reservation. >> >> bndtools -- is an eclipse plugin that is very helpful when developing >> OSGi bundles. It does a lot of grimy, boilerplate things for you. >> >> inlining -- is where one places dependent *.jar files inside the OSGi >> bundle and therefore on said bundle's class path. It tends to promote >> bloated bundles--not in the spirit of OSGi--but sometimes necessary. >> >> componentizing -- is the business of converting a class into a component. >> In the bndtools way of doing things, this can be a easy as annotating a >> class with @Component. >> >> bundle -- You probably know what this is already, but I'll include it for >> good measure. A bundle is a body of code that is on the same class path, >> and often acts as a service to there bundles. >> >> I don't know what could be done upstream other that making Accumulo's >> client OAGI ready. Would we like to do that? >> >> >> On Mon, Apr 7, 2014 at 11:02 AM, Josh Elser wrote: >> >>> You just used a lot of words that don't mean anything to me :) >>> >>> Hopefully you don't have to do much on your own. If there are things we >>> can change upstream to make this process easier, please feel free to let us >>> know. >>> >>> >>> On 4/7/14, 10:55 AM, Geoffry Roberts wrote: >>> >>>> Thanks Josh, >>>> >>>> My container for the moment is equinox, but all should work in Felix as >>>> well. I've been using bndtools for my other OSGi work so I'm faced with >>>> either annotating the Accumulo Code or wrapping it somehow. What do you >>>> want to bet I wind up inlining it? Still, the annotated (read >>>> componentized) approach would be less kloogy. I hesitate because I'd >>>> wind up maintaining my own code line. >>>> >>>> >>>> On Mon, Apr 7, 2014 at 10:28 AM, Josh Elser >>> > wrote: >>>> >>>> On 4/7/14, 10:07 AM, Geoffry Roberts wrote: >>>> >>>> My original question remains: Is the Accumulo Client dependent >>>> on the >>>> Hadoop Client fully? This determination can be made through >>>> trial and >>>> error. But I'm looking to leverage OPE (other people's >>>> experience) if >>>> it exists. >>>> >>>> >>>> I thought someone had already said this (but I may be confusing >>>> threads): the Accumulo API uses Text throughout. Hadoop is a >>>> required dependency. >>>> >>>> >>>> In the same spirit, does anyone know if all the following are >>>> required >>>> to run an Accumulo Client? core, fate, start, trace? If I >>>> attempt to >>>> OSGify, I'm trying to figure how much trouble am I getting into. >>>> >>>> >>>> Yes, that should be about it from within Accumulo. You might need >>>> some other foss dependencies also available, but I'm not aware on >>>> what your "container" (or w/e the proper terminology would be) >>>> provides. >>>> >>>> >>>> >>>> >>>> -- >>>> There are ways and there are ways, >>>> >>>> Geoffry Roberts >>>> >>> >> >> >> -- >> There are ways and there are ways, >> >> Geoffry Roberts >> > > -- There are ways and there are ways, Geoffry Roberts --f46d043c086449d30a04f69e0c17 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

= Corey,


Interesting you have Hadoop working in Karaf. =A0I'm using equinox. = =A0It also sounds as if I don't need to access HFDS in order to get Acc= umulo to work in OSGi. If I understand=A0you=A0correctly, I only need have= Text available. =A0I'll look=A0i= nto=A0that. It does answer my questio= n and maybe I can avoid the JAAS mishmash.


= I have been using OSGi off and on for a few years now. Close on to 100% of = my troubles has always been with getting 3rd party libraries working. =A0Ac= cumulo/Hadoop is only the latest round. =A0


I'm gearing to do s= ome major work based on Accumulo. =A0It's security, scaleability, relat= ionship to Hadoop's HDFS and MR all conspire to make it attractive. =A0= But creating an uberbundle? I'm sure I could get it working as a proof = of concept, but will it play in=A0prime-time?


How hard woul= d it be to make a proper Accumulo bundle? =A0Would the community accept it?=


Thanks=


So far as my trava= ils with JAAS is=A0concerned,=A0I did this in my bndtools *.bndrun file:


-runproperties:=A0org.osgi.framework.system.packages.extra=3Dcom.sun.security.auth.module<= /span>


I also tried:=A0
Import-Package:=A0com.sun.security.auth.module=A0
in my bu= ndle that calls Hadoop.

<= /font>
Both of these were advised by one of the bndtools gu= rus--neither worked. =A0When I did the Import-Package other things broke.

On Wed, Apr 9, 20= 14 at 9:46 AM, Corey Nolet <cjnolet@gmail.com> wrote:
Geoffry,

As Josh pointed out, you shoul= d only need the Hadoop libraries on the client side to use the Text object.= This means you won't have to go through the pain of placing the xml fi= les in your root bundles.=A0

Did you try the JAAS export from the packages in your c= ontainer? Did that help?

I agree with your com= ment that we *should* be able to look at these bundles as black boxes but w= e're not dealing with true bundles, yet. Once I got my Hadoop client bu= ndle ready to go, I haven't had to touch it (and the Hadoop Karaf proje= ct solved most of the import/export guessing work for me already). =A0For A= ccumulo, on the other hand, it's up to you if you want to create an ube= r-bundle with the necessary Accumulo packages or just wrap each one and pro= vide the necessary imports/exports across them. For my needs, I just create= d one uber-bundle for Accumulo packages and imported the necessary non-Accu= mulo packages (many of those I had to wrap as well). I didn't find that= part too painful on the Accumulo side. While not being the ideal situation= from the OSGI side, with lack of an existing bundle artifact, it did solve= my problem and I'm actively using both Accumulo and Hadoop in OSGi. My= recommendation would be that until we get a proper bundle, that solution w= ould certainly work in the short-term.=A0

I believe Josh posted this already but check out [1]. A= ready-to-go OSGi bundle for Accumulo would be useful but the Hadoop client= dependency would need to be wrapped (or exposed as its own bundle). IMO, w= ith proper documentation this shouldn't be too painful for users. Thoug= hts?




On Mon, Apr 7, 2014 at 11:27 AM, Geoffry Roberts <threadedblue@gmail.= com> wrote:
Ahh, =A0let me try and address where I might have gone off= the linguistic reservation.

bndtools -- is an eclipse p= lugin that is very helpful when developing OSGi bundles. =A0It does a lot o= f grimy, boilerplate things for you.

inlining -- is where one places dependent *.jar files i= nside the OSGi bundle and therefore on said bundle's class path. =A0It = tends to promote bloated bundles--not in the spirit of OSGi--but sometimes = necessary.

componentizing -- is the business of converting a class= into a component. =A0In the bndtools way of doing things, this can be a ea= sy as annotating a class with @Component. =A0

bund= le -- You probably know what this is already, but I'll include it for g= ood measure. =A0A bundle is a body of code that is on the same class path, = and often acts as a service to there bundles.

I don't know what could be done upstream other that= making Accumulo's client OAGI ready. =A0Would we like to do that?


On Mon, Apr 7, 2014 at 11:02 AM, Josh Elser <josh.elser@gmail.com> wrote:
You just used a lot of words that don't mean anything = to me :)

Hopefully you don't have to do much on your own. If there are things we= can change upstream to make this process easier, please feel free to let u= s know.


On 4/7/14, 10:55 AM, Geoffry Roberts wrote:
<mailto:josh.e= lser@gmail.com>> wrote:

=A0 =A0 On 4/7/14, 10:07 AM, Geoffry Roberts wrote:

=A0 =A0 =A0 =A0 My original question remains: Is the Accumulo Client depend= ent
=A0 =A0 =A0 =A0 on the
=A0 =A0 =A0 =A0 Hadoop Client fully? =A0This determination can be made thro= ugh
=A0 =A0 =A0 =A0 trial and
=A0 =A0 =A0 =A0 error. =A0But I'm looking to leverage OPE (other people= 's
=A0 =A0 =A0 =A0 experience) if
=A0 =A0 =A0 =A0 it exists.


=A0 =A0 I thought someone had already said this (but I may be confusing
=A0 =A0 threads): the Accumulo API uses Text throughout. Hadoop is a
=A0 =A0 required dependency.


=A0 =A0 =A0 =A0 In the same spirit, does anyone know if all the following a= re
=A0 =A0 =A0 =A0 required
=A0 =A0 =A0 =A0 to run an Accumulo Client? core, fate, start, trace? =A0If = I
=A0 =A0 =A0 =A0 attempt to
=A0 =A0 =A0 =A0 OSGify, I'm trying to figure how much trouble am I gett= ing into.


=A0 =A0 Yes, that should be about it from within Accumulo. You might need =A0 =A0 some other foss dependencies also available, but I'm not aware = on
=A0 =A0 what your "container" (or w/e the proper terminology woul= d be) provides.




--
There are ways and there are ways,

Geoffry Roberts



--
There are ways and there are ways,=A0

Geoffry= Roberts




--
=
There are ways and there are ways,=A0

<= /div>Geoffry Roberts
--f46d043c086449d30a04f69e0c17--