accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Billie Rinaldi <billie.rina...@gmail.com>
Subject Re: GSoC proposal: Ambari for Accumulo - Final Draft
Date Wed, 01 May 2013 16:16:54 GMT
I think including it in the proposal isn't necessary.  I just wanted to
give you a heads up.  :-)

Billie


On Wed, May 1, 2013 at 7:28 AM, Andres Danter <adanter@gmail.com> wrote:

> Thanks, Billie.  I agree about working with 1.5.0.  I was going to be
> sneaky and try to integrate with both versions, but I didn't want to put it
> on the proposal in case I run into complications.  Let me know if you want
> me to include that in the proposal, though, and I'll add it.
>
> Thanks again.
>
> Andres
>
>
> On Wed, May 1, 2013 at 10:20 AM, Billie Rinaldi <billie.rinaldi@gmail.com
> >wrote:
>
> > Andres,
> >
> > The proposal looks good.  I believe Accumulo will work with Hadoop 1.0.4.
> > You may want to work both with Accumulo 1.5.0 and 1.4.3 if we get 1.5.0
> > released soon enough.
> >
> > Billie
> >
> >
> > On Tue, Apr 30, 2013 at 10:25 PM, Andres Danter <adanter@gmail.com>
> wrote:
> >
> > > Hello Ambari and Accumulo developers,
> > >
> > > Here is the link to the final draft of my proposal.  Not too much has
> > > changed.  Some of the language on the previous draft was confusing, so
> I
> > > cleaned that up, and I did add some Ambari tidbits.  The biggest
> > addition,
> > > in my opinion, is the table that lists the different versions of
> > software I
> > > will be using.  Please take a look at that table to ensure I don't
> have a
> > > mismatch.  I have a nagging feeling that the version of Hadoop that
> > Ambari
> > > can deploy is newer than the version Accumulo 1.4.3 is comfortable
> with.
> > >
> > > Thank you all again for your support and feedback.  Here is the link:
> > >
> > >
> > >
> >
> https://docs.google.com/file/d/0BxBY29P7A1RgVnZscGcycExDbEU/edit?usp=sharing
> > >
> > >
> > > Andres
> > >
> > >
> > > On Mon, Apr 29, 2013 at 8:03 PM, Andres Danter <adanter@gmail.com>
> > wrote:
> > >
> > > > Hi Drew,
> > > >
> > > > Sorry for replying so late.  Unfortunately, this project is a solo
> > thing.
> > > > Hopefully we can team up on another project in the future.
> > > >
> > > > Thanks for the support.
> > > >
> > > > Andres
> > > >
> > > >
> > > > On Fri, Apr 26, 2013 at 11:01 AM, Drew Pierce <drewpierce@live.com>
> > > wrote:
> > > >
> > > >> Not sure if these are solo initiatives or those needing assistance.
> > Let
> > > >> me know if it is the latter :)
> > > >>
> > > >>
> > > >> > Date: Fri, 26 Apr 2013 10:47:21 -0400
> > > >> > Subject: Re: GSoC proposal: Ambari for Accumulo - forever : )
> > > >> > From: adanter@gmail.com
> > > >> > To: dev@accumulo.apache.org
> > > >> >
> > > >> > Thanks, Drew, for the enthusiam.
> > > >> >
> > > >> > And thanks, John, for the good information.  It is definitely
an
> > area
> > > I
> > > >> > need to look into.
> > > >> >  On Apr 26, 2013 10:31 AM, "John Vines" <vines@apache.org>
wrote:
> > > >> >
> > > >> > > The init.d scripts we provide are well tested, provided
you have
> > > >> accumulo
> > > >> > > installed into /usr/lib. There is probably room for improvement
> to
> > > >> utilize
> > > >> > > /etc/defaults/accumulo to make it configurable. This makes
them
> > not
> > > >> > > configurable with the existing RPM packaging internal to
> Accumulo.
> > > And
> > > >> > > Ambari uses the init.d scripts to start/stop things.
> > > >> > >
> > > >> > > As mentioned in the other thread, I'm pretty sure Ambari
is
> pretty
> > > >> tightly
> > > >> > > integrated with Bigtop. There is an Accumulo ticket for
bigtop
> > > >> support (
> > > >> > > Accumulo-138 ). I went ahead and threw a patch on there
in which
> > the
> > > >> RPM
> > > >> > > generated has been tested (the deb is lacking though).
> > Furthermore,
> > > >> they do
> > > >> > > not use the given Accumulo init.d scripts, so there may
be room
> > for
> > > >> > > improvement in there somewhere, but I'm not sure.
> > > >> > >
> > > >> > > Should you need to work down that path more, I can definitely
> > > provide
> > > >> more
> > > >> > > guidance.
> > > >> > >
> > > >> > >
> > > >> > > On Thu, Apr 25, 2013 at 12:18 PM, Keith Turner <
> keith@deenlo.com>
> > > >> wrote:
> > > >> > >
> > > >> > > > On Thu, Apr 25, 2013 at 11:40 AM, Andres Danter <
> > > adanter@gmail.com>
> > > >> > > wrote:
> > > >> > > >
> > > >> > > > > Excellent feedback, Keith. Thanks.
> > > >> > > > >
> > > >> > > > > I should have mentioned the REST API in the proposal.
 I'm
> not
> > > >> sure if
> > > >> > > I
> > > >> > > > > can use it for the CLI.  I will certainly try,
since I would
> > > like
> > > >> to
> > > >> > > > adhere
> > > >> > > > > to the Ambari architecture as much as possible.
> > > >> > > > >
> > > >> > > > > As for your other points:
> > > >> > > > >
> > > >> > > > > *3.1.1 loggers do not exists in 1.5 ... so would
only need
> to
> > > >> consider
> > > >> > > > them
> > > >> > > > > if working w/ 1.4 .  I suppose you would want
to work
> Accumulo
> > > >> 1.4,
> > > >> > > since
> > > >> > > > > that the current stable release.*
> > > >> > > > >
> > > >> > > > > Yes. I was going to work with Accumulo 1.4, but
it is good
> to
> > > >> know that
> > > >> > > > the
> > > >> > > > > loggers are not supported in 1.5.
> > > >> > > > >
> > > >> > > > > *3.1.3
> > > >> > > > >   Accumulo supports reading system config from
xml local
> file
> > > and
> > > >> then
> > > >> > > > > zookeeper (whats set in zookeeper takes precedence).
 The
> nice
> > > >> thing
> > > >> > > > about
> > > >> > > > > setting something a config zookeeper, is that
process
> restart
> > is
> > > >> not
> > > >> > > > > required.*
> > > >> > > > >
> > > >> > > > > Good to know this too.  Does this only affect
Zookeeper?  I
> > > should
> > > >> > > > > distinguish between processes that may need restarts
to
> load a
> > > >> > > > > configuration vs those that do not.  Is the Zookeeper
> process
> > > the
> > > >> only
> > > >> > > > one
> > > >> > > > > like that?
> > > >> > > > >
> > > >> > > > >
> > > >> > > > Not sure about zookeeper, I think it requires a restart.
> >  Accumulo
> > > >> can
> > > >> > > > optionally store a lot of its configuration in zookeeper.
>  When
> > > >> Accumulo
> > > >> > > > configuration stored in zookeeper is changed, Accumulo
> processes
> > > >> may pick
> > > >> > > > up these config changes immediately w/o restart.  
Some
> > properties
> > > >> do
> > > >> > > > require a restart when changed in zookeeper.  This
is
> documented
> > > in
> > > >> > > > Accumulo config documentation.
> > > >> > > >
> > > >> > > >
> > > >> > > > >
> > > >> > > > > *For section 3.2, are you proposing adding features
to the
> > > >> existing
> > > >> > > > Ambari
> > > >> > > > > GUI?  If so, are the Accumulo specific?*
> > > >> > > > >
> > > >> > > > > I am proposing to add features to the existing
Ambari GUI.
> >  The
> > > >> idea
> > > >> > > here
> > > >> > > > > is that something like user access controls can
be
> generalized
> > > so
> > > >> that
> > > >> > > > any
> > > >> > > > > Hadoop component being managed by Ambari is able
to utilize
> > > them.
> > > >>  I'm
> > > >> > > > > hoping that the Ambari PMC is OK with me doing
this.  I do
> not
> > > >> want to
> > > >> > > > > introduce any Accumulo-specific features that
would make no
> > > sense
> > > >> for
> > > >> > > > other
> > > >> > > > > components.
> > > >> > > > > *
> > > >> > > > > It seems that Ambari relies on RPMs.  Its possible
you may
> > have
> > > to
> > > >> > > spend
> > > >> > > > > time creating Accumulo RPMs that are useful for
Ambari. You
> > may
> > > >> want to
> > > >> > > > > think about allocating some time for this.  Christopher
and
> > John
> > > >> can
> > > >> > > give
> > > >> > > > > you more info about the current state of Accumulo
RPMs.*
> > > >> > > > >
> > > >> > > > > This is a very good point.  While building RPMs
is not
> > > difficult,
> > > >> it
> > > >> > > does
> > > >> > > > > require time to setup the configuration and test
the builds.
> >  I
> > > >> will
> > > >> > > > > include time for this in the proposal.
> > > >> > > > >
> > > >> > > >
> > > >> > > > I suppose I am really thinking about are scripts for
starting
> > and
> > > >> > > stopping
> > > >> > > > Accumulo.  For an RPM to be useful it should provide
good
> > > >> /etc/init.d
> > > >> > > > scripts for starting and stopping Accumulo processes.
  I am
> not
> > > >> sure
> > > >> > > what
> > > >> > > > the status of these scripts is in the various RPMs
built by
> > > >> Accumulo.
> > > >> > >  This
> > > >> > > > may not even be a concern, I am not sure how Ambari
starts and
> > > stop
> > > >> > > > services.   If you have not already, its something
to consider
> > > when
> > > >> > > > scheduling.
> > > >> > > >
> > > >> > > >
> > > >> > > > >
> > > >> > > > > Thanks again, Keith.
> > > >> > > > >
> > > >> > > > > Andres
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > On Thu, Apr 25, 2013 at 10:54 AM, Keith Turner
<
> > > keith@deenlo.com>
> > > >> > > wrote:
> > > >> > > > >
> > > >> > > > > > About the REST API, I meant to ask if this
could be used
> > from
> > > >> the
> > > >> > > > command
> > > >> > > > > > line currently?  Or is too cumbersome?
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > On Thu, Apr 25, 2013 at 10:09 AM, Andres
Danter <
> > > >> adanter@gmail.com>
> > > >> > > > > wrote:
> > > >> > > > > >
> > > >> > > > > > > Yup.  Looks like that's the way to go.
 Here is the link
> > to
> > > >> the
> > > >> > > > > document.
> > > >> > > > > > > Let me know if you have any problems
accessing it.
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >>
> > >
> >
> https://docs.google.com/file/d/0BxBY29P7A1RgVnZscGcycExDbEU/edit?usp=sharing
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > On Thu, Apr 25, 2013 at 9:51 AM, Keith
Turner <
> > > >> keith@deenlo.com>
> > > >> > > > > wrote:
> > > >> > > > > > >
> > > >> > > > > > > > Nothing.  I suppose the mailing
list is stripping it
> > off.
> > > >>  Could
> > > >> > > > post
> > > >> > > > > > it
> > > >> > > > > > > as
> > > >> > > > > > > > an attachment to the ticket or
put it on google drive
> or
> > > >> dropbox
> > > >> > > > and
> > > >> > > > > > > send a
> > > >> > > > > > > > link.
> > > >> > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > > On Thu, Apr 25, 2013 at 9:44 AM,
Andres Danter <
> > > >> > > adanter@gmail.com>
> > > >> > > > > > > wrote:
> > > >> > > > > > > >
> > > >> > > > > > > > > Weird.  I see the attachment
on the email I sent,
> > which
> > > >> means
> > > >> > > > that
> > > >> > > > > it
> > > >> > > > > > > > > might have been stripped off.
 Here it is again.
>  Let
> > me
> > > >> know
> > > >> > > if
> > > >> > > > > you
> > > >> > > > > > > > still
> > > >> > > > > > > > > don't see it.
> > > >> > > > > > > > >
> > > >> > > > > > > > > Thanks,
> > > >> > > > > > > > >
> > > >> > > > > > > > > Andres
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > > On Thu, Apr 25, 2013 at 9:18
AM, Keith Turner <
> > > >> > > keith@deenlo.com>
> > > >> > > > > > > wrote:
> > > >> > > > > > > > >
> > > >> > > > > > > > >> was there supposed to
be an attachment?
> > > >> > > > > > > > >>
> > > >> > > > > > > > >>
> > > >> > > > > > > > >> On Thu, Apr 25, 2013 at
2:19 AM, Andres Danter <
> > > >> > > > adanter@gmail.com
> > > >> > > > > >
> > > >> > > > > > > > wrote:
> > > >> > > > > > > > >>
> > > >> > > > > > > > >> > Hi Billie,
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > Here is a draft of
my proposal.  I've yet to
> write
> > > the
> > > >> > > > technical
> > > >> > > > > > > > >> approach
> > > >> > > > > > > > >> > section (have not
idea what I'm going to put
> there)
> > > >> and the
> > > >> > > > > "about
> > > >> > > > > > > me"
> > > >> > > > > > > > >> > section.  I definitely
appreciate any feedback
> from
> > > >> you and
> > > >> > > > > anyone
> > > >> > > > > > > in
> > > >> > > > > > > > >> the
> > > >> > > > > > > > >> > development group.
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > Thank you,
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > Andres
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >>
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >>
> > > >>
> > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message