accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andres Danter <adan...@gmail.com>
Subject Re: GSOC Idea - Accumulo Control Panel
Date Sat, 13 Apr 2013 19:20:41 GMT
Thanks for the suggestion, David.  I'm not familiar with Yammer's metrics
libraries.  I would like to keep this project limited to Ambari and
Accumulo, but I will look at the links you included and learn more about
that library, for future consideration.

Thanks again,

Andres


On Sat, Apr 13, 2013 at 1:44 PM, David Schlosnagle <schlosna@gmail.com>wrote:

> In regards to #6, I'm curious if you've considered using Yammer's Apache
> licensed metrics library [1] for capturing stats as it offers many
> different options for consuming and monitoring produced metrics. Coda Hale
> gives a good overview in this talk [2].
>
> [1]: http://metrics.codahale.com/
> [2]: http://pivotallabs.com/139-metrics-metrics-everywhere/
>
> - Dave
>
> On Apr 13, 2013, at 12:15 PM, Andres Danter <adanter@gmail.com> wrote:
>
> > Awesome suggestions!  Thank you, Josh.
> > On Apr 12, 2013 10:01 PM, "Josh Elser" <josh.elser@gmail.com> wrote:
> >
> >> Andres,
> >>
> >> Looks good. Glad to see a lot of good ideas here already. I have a
> couple
> >> of refinements if you don't mind.
> >>
> >> 3. Have the ability to schedule /graceful/ restart of Accumulo...
> >>
> >> It would be really cool to have the option of bringing down an entire
> >> cluster or do a rolling-restart (iteratively bring down a node, let
> things
> >> quiesce, then restart that node) to try to ensure 100% availability.
> >> Accumulo handles most of this for you -- it's more a matter of waiting
> for
> >> any unhosted tablets to be re-assigned after a tabletserver is stopped.
> >>
> >> 4. Push Accumulo code ...
> >>
> >> With 1.5 releases, there's a VFS classloader built in which allows for
> >> various types of file systems to be used to load files onto the
> classpath.
> >> The most interesting of which probably include HDFS, HTTP(S), and
> (S)FTP.
> >> It would be neat to support not only loading classes from the local
> >> filesystem but also by a variety of other means.
> >>
> >> 6. Gather system and/or application metrics..
> >>
> >> Just be aware that this is a very open-ended subject and, most likely,
> >> could be a full GSOC task on its own. You probably want to think about
> what
> >> might be best for Accumulo (starting with what the monitor and master
> >> already supply) and how best you can integrate with Ambari.
> >>
> >> Again, overall, sounds awesome. I look forward to working with you.
> >>
> >>
> >> On 04/12/2013 05:58 PM, Andres Danter wrote:
> >>
> >>> Thanks, Billie.  I'm liking this task more and more.  I would love to
> take
> >>> this on for GSoC.  Please run it by the Ambari PMC.  I will start
> drafting
> >>> the proposal this weekend.
> >>>
> >>> Just to be clear, the goal would be to integrate Ambari with Accumulo
> in
> >>> order to achieve the following capabilities (I've expanded David's
> >>> original
> >>> list of features):
> >>>
> >>> 1. Control (start/stop) Accumulo processes on any cluster node
> >>> 2. Have the ability to edit Accumulo configuration files via a
> web-based
> >>> editor (I don't think Ambari provides this.  Please let me know if I'm
> >>> wrong)
> >>> 3. Have the ability to schedule restart of Accumulo processes after a
> >>> configuration change
> >>> 4. Push Accumulo code and configuration files to a new cluster node
> >>> 5.  Gracefully remove a node from a cluster
> >>> 6.  Gather system and/or application metrics that can be used to
> monitor
> >>> the system and perhaps create reports on usability
> >>> 7.  Be able to monitor and control the Accumulo cluster from a
> web-based
> >>> GUI  (this kind of goes with #2 above)
> >>> 8.  Allow read-only monitor sessions, as well as full-admin sessions
> and
> >>> have the ability to strictly manage who can access either.  (this I got
> >>> from ACCUMULO-196)
> >>> 9.  Provide the ability to authenticate user sessions with SSL
> >>> (see ACCUMULO-196)
> >>>
> >>> It seems like the product of this work would largely replace the
> current
> >>> Accumulo Monitor.  If that is the case, are there features of the
> monitor
> >>> which you would like to see integrated?
> >>>
> >>> I will have about 3 months to complete this work (actually about
> 220-240
> >>> hours), so that list is doable.  Most of the work will involve the GUI
> >>> portions, unless Ambari already comes with much of the functionality.
>  If
> >>> the list is too short, please let me know.
> >>>
> >>> I guess I should sign up to the Ambari mailing list as well, right?
> >>>
> >>> Thanks again and I look forward to being your mentee and to working
> with
> >>> the Accumulo community.
> >>>
> >>> Andres
> >>>
> >>>
> >>> On Fri, Apr 12, 2013 at 3:43 PM, David Medinets <
> david.medinets@gmail.com
> >>>> **wrote:
> >>>
> >>> The project is open source ... we can't stop you!
> >>>>
> >>>>
> >>>> On Fri, Apr 12, 2013 at 2:25 PM, Andres Danter <adanter@gmail.com>
> >>>> wrote:
> >>>>
> >>>> Could the work of integrating Accumulo with Ambari be something that
> you
> >>>>> would allow a student to tackle for GSoC?  I don't want to step
on
> >>>> anyone's
> >>>>
> >>>>> planned work, but such a task really interests me.  I want to tackle
> >>>>> something with some meat, if you know what I mean.
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Fri, Apr 12, 2013 at 11:47 AM, David Medinets
> >>>>> <david.medinets@gmail.com>**wrote:
> >>>>>
> >>>>> Thanks for the reminder. I had forgotten the name of Ambari.
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Apr 12, 2013 at 10:37 AM, Keith Turner <keith@deenlo.com>
> >>>>> wrote:
> >>>>
> >>>>> On Fri, Apr 12, 2013 at 10:29 AM, David Medinets
> >>>>>>> <david.medinets@gmail.com> wrote:
> >>>>>>>
> >>>>>>>> It might be interesting to have a tool which could:
> >>>>>>>>
> >>>>>>>> a) shut down any accumulo process on any server.
> >>>>>>>> b) start any accumulo process on any server.
> >>>>>>>> c) edit the various configuration files through a web
interface.
> >>>>>>>> d) automate restart accumulo processes, as needed, after
> >>>>>>> configuration
> >>>>>
> >>>>>> change.
> >>>>>>>> e) push accumulo code and configuration to a new server.
> >>>>>>>> f) gracefully shutdown and remove a server from the
cluster.
> >>>>>>>>
> >>>>>>>> Anything else? If this sounds like a good idea, let's
create a
> JIRA
> >>>>>>> ticket
> >>>>>>>
> >>>>>>>> with the right tags.
> >>>>>>> This sounds like Apache Ambari, which we should possibly
integrate
> >>>>>>> with. See ACCUMULO-136
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message