incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andreas Neumann <a...@apache.org>
Subject Re: Hoya Proposal
Date Tue, 14 Jan 2014 03:33:55 GMT
Hi Steve,

Hoya has a lot of good ideas. How do think it relates to Twill (recently
incubated at http://twill.incubator.apache.org/)? I feel that there is a
lot of overlap in what both projects are trying to achieve. Do you think
they complement each other? Is there opportunity for collaboration, perhaps
cross-contributing or even converging the two projects?

-Andreas.


On Wed, Jan 8, 2014 at 7:08 AM, Steve Loughran <stevel@hortonworks.com>wrote:

> I'm starting to put together the incubation proposal for Hoya: a tool to
> dynamically deploy applications such as HBase or Accumulo on YARN
>
> https://wiki.apache.org/incubator/HoyaProposal
>
> It does already work to the extent that it can bring up either application,
> run different clusters of different versions, and remember where containers
> were allocated so that on application restart it can ask for them back.
> That increases data locality and makes a big difference with HBase.
>
> It also needs a lot more work -YARN-896 is adding YARN features that help,
> but there's lots of fun to be had in Hoya including
>  -leading edge work in failure handling, modelling cluster unreliability
> and reacting to it. Can we move beyond simple blacklisting to
> "greylisting", accepting unreliable boxes if we have no altenatives
>
> Then there's adding more providers, to support different application
> installations -I'm starting to write a functional test framework which need
> provider-specific workload generations
>
> Other features: AM should have a web ui that redirects to the live
> endpoints to all the app-specific UIs (e.g. HBase Master GUI), as well as
> displaying cluster state itself, for people and for management tools....
>
> To summarise: lots of fun to be had
>
> -steve
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message