incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: Hoya Proposal
Date Wed, 15 Jan 2014 18:40:14 GMT
I wanted to weigh in on some of Steve's thoughts, I'm actually really
excited about being able to leverage Hoya inside of Accumulo itself.

We presently have a few system tests that rely on manual set up, which can
be frustrating to deal with on a beefy boxes (running multiple Accumulo
procs on a single host). Hoya drastically reduces the amount of effort it
takes to run these distributed tests in a 'closer to real' environment.
On Jan 15, 2014 8:36 AM, "Steve Loughran" <stevel@hortonworks.com> wrote:

> On 15 January 2014 02:13, Andreas Neumann <anew@apache.org> wrote:
>
> > I see. So is Hoya limited to HBase and Accumulo? Or is it open for any
> > other type of existing application? If so, won't it have some common
> > abstraction that is shared by all of them? That is where I see the
> > similarity with Twill.
> >
> >
> it started off as pure hbase, but now has the notion of a provider which
> has a client-side and server side
>
> client side
>  -helps set up the template JSON file to describe the cluster (e.g. adds
> default values),
>  -patches the configuration directory supplied at creation time with
> whatever options it wants (e.g links up fs.default.name & ZK settings in
> hbase-site.xml)
>  -does preflight validation of cluster options
>  -can also add its own .tar.gz to the resources of the AM (or any other
> resources)
>
> server side
>  -runs in the AM and sets up everything needed to run instances of a role
> (e.g. HBase master, Accumulo GC, ..)
>  -can run code in the AM to help set things up (e.g. Accumulo provider
> service runs bin/accumulo init if needed)
>  -TODO: liveness monitoring
>
> the requirements of an app to be deployable are
>
> https://github.com/hortonworks/hoya/blob/master/src/site/markdown/app_needs.md
>
>
>
> > Whereas, if it is a separate effort for each existing application, say
> > HBase, then what is the end goal for Hoya when it comes out of
> incubation?
> > To merge it back into HBase proper?
> >
> >
> Now that it supports >1 application, it can't go into HBase. The individual
> provider services can (their implementation classes are worked out via the
> configuration.XML).
>
> But as we use the accumulo and HBase apps for testing, its really good to
> have them both in the hoya project right now -a project that builds
> downstream of both and needs
> to be given the Hadoop filesystem paths to .tar.gz files of each app.
>
> There's nothing to stop 3rd party apps joining in, indeed, one thing I'd
> like someone to do is actually have Hoya deploy SmartFrog .tar.gz files and
> pass down configurations that deploy applications -for example jetty-based
> web servers. That'd take an intern working at HP Labs over the summer,
> maybe
>
> -steve
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message