accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Schlosnagle <schlo...@gmail.com>
Subject Re: GSOC Idea - Accumulo Control Panel
Date Sat, 13 Apr 2013 17:44:55 GMT
In regards to #6, I'm curious if you've considered using Yammer's Apache licensed metrics library
[1] for capturing stats as it offers many different options for consuming and monitoring produced
metrics. Coda Hale gives a good overview in this talk [2].

[1]: http://metrics.codahale.com/
[2]: http://pivotallabs.com/139-metrics-metrics-everywhere/

- Dave

On Apr 13, 2013, at 12:15 PM, Andres Danter <adanter@gmail.com> wrote:

> Awesome suggestions!  Thank you, Josh.
> On Apr 12, 2013 10:01 PM, "Josh Elser" <josh.elser@gmail.com> wrote:
> 
>> Andres,
>> 
>> Looks good. Glad to see a lot of good ideas here already. I have a couple
>> of refinements if you don't mind.
>> 
>> 3. Have the ability to schedule /graceful/ restart of Accumulo...
>> 
>> It would be really cool to have the option of bringing down an entire
>> cluster or do a rolling-restart (iteratively bring down a node, let things
>> quiesce, then restart that node) to try to ensure 100% availability.
>> Accumulo handles most of this for you -- it's more a matter of waiting for
>> any unhosted tablets to be re-assigned after a tabletserver is stopped.
>> 
>> 4. Push Accumulo code ...
>> 
>> With 1.5 releases, there's a VFS classloader built in which allows for
>> various types of file systems to be used to load files onto the classpath.
>> The most interesting of which probably include HDFS, HTTP(S), and (S)FTP.
>> It would be neat to support not only loading classes from the local
>> filesystem but also by a variety of other means.
>> 
>> 6. Gather system and/or application metrics..
>> 
>> Just be aware that this is a very open-ended subject and, most likely,
>> could be a full GSOC task on its own. You probably want to think about what
>> might be best for Accumulo (starting with what the monitor and master
>> already supply) and how best you can integrate with Ambari.
>> 
>> Again, overall, sounds awesome. I look forward to working with you.
>> 
>> 
>> On 04/12/2013 05:58 PM, Andres Danter wrote:
>> 
>>> Thanks, Billie.  I'm liking this task more and more.  I would love to take
>>> this on for GSoC.  Please run it by the Ambari PMC.  I will start drafting
>>> the proposal this weekend.
>>> 
>>> Just to be clear, the goal would be to integrate Ambari with Accumulo in
>>> order to achieve the following capabilities (I've expanded David's
>>> original
>>> list of features):
>>> 
>>> 1. Control (start/stop) Accumulo processes on any cluster node
>>> 2. Have the ability to edit Accumulo configuration files via a web-based
>>> editor (I don't think Ambari provides this.  Please let me know if I'm
>>> wrong)
>>> 3. Have the ability to schedule restart of Accumulo processes after a
>>> configuration change
>>> 4. Push Accumulo code and configuration files to a new cluster node
>>> 5.  Gracefully remove a node from a cluster
>>> 6.  Gather system and/or application metrics that can be used to monitor
>>> the system and perhaps create reports on usability
>>> 7.  Be able to monitor and control the Accumulo cluster from a web-based
>>> GUI  (this kind of goes with #2 above)
>>> 8.  Allow read-only monitor sessions, as well as full-admin sessions and
>>> have the ability to strictly manage who can access either.  (this I got
>>> from ACCUMULO-196)
>>> 9.  Provide the ability to authenticate user sessions with SSL
>>> (see ACCUMULO-196)
>>> 
>>> It seems like the product of this work would largely replace the current
>>> Accumulo Monitor.  If that is the case, are there features of the monitor
>>> which you would like to see integrated?
>>> 
>>> I will have about 3 months to complete this work (actually about 220-240
>>> hours), so that list is doable.  Most of the work will involve the GUI
>>> portions, unless Ambari already comes with much of the functionality.  If
>>> the list is too short, please let me know.
>>> 
>>> I guess I should sign up to the Ambari mailing list as well, right?
>>> 
>>> Thanks again and I look forward to being your mentee and to working with
>>> the Accumulo community.
>>> 
>>> Andres
>>> 
>>> 
>>> On Fri, Apr 12, 2013 at 3:43 PM, David Medinets <david.medinets@gmail.com
>>>> **wrote:
>>> 
>>> The project is open source ... we can't stop you!
>>>> 
>>>> 
>>>> On Fri, Apr 12, 2013 at 2:25 PM, Andres Danter <adanter@gmail.com>
>>>> wrote:
>>>> 
>>>> Could the work of integrating Accumulo with Ambari be something that you
>>>>> would allow a student to tackle for GSoC?  I don't want to step on
>>>> anyone's
>>>> 
>>>>> planned work, but such a task really interests me.  I want to tackle
>>>>> something with some meat, if you know what I mean.
>>>>> 
>>>>> 
>>>>> 
>>>>> On Fri, Apr 12, 2013 at 11:47 AM, David Medinets
>>>>> <david.medinets@gmail.com>**wrote:
>>>>> 
>>>>> Thanks for the reminder. I had forgotten the name of Ambari.
>>>>>> 
>>>>>> 
>>>>>> On Fri, Apr 12, 2013 at 10:37 AM, Keith Turner <keith@deenlo.com>
>>>>> wrote:
>>>> 
>>>>> On Fri, Apr 12, 2013 at 10:29 AM, David Medinets
>>>>>>> <david.medinets@gmail.com> wrote:
>>>>>>> 
>>>>>>>> It might be interesting to have a tool which could:
>>>>>>>> 
>>>>>>>> a) shut down any accumulo process on any server.
>>>>>>>> b) start any accumulo process on any server.
>>>>>>>> c) edit the various configuration files through a web interface.
>>>>>>>> d) automate restart accumulo processes, as needed, after
>>>>>>> configuration
>>>>> 
>>>>>> change.
>>>>>>>> e) push accumulo code and configuration to a new server.
>>>>>>>> f) gracefully shutdown and remove a server from the cluster.
>>>>>>>> 
>>>>>>>> Anything else? If this sounds like a good idea, let's create
a JIRA
>>>>>>> ticket
>>>>>>> 
>>>>>>>> with the right tags.
>>>>>>> This sounds like Apache Ambari, which we should possibly integrate
>>>>>>> with. See ACCUMULO-136
>> 

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message