incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Baptiste Onofré ...@nanthrax.net>
Subject Re: Hoya Proposal
Date Wed, 08 Jan 2014 15:18:34 GMT
Hi Steve,

it looks interesting. It's a kind of "advanced" provisioning for Hadoop 
cluster. I see some potential interesting "work together" with other 
projects.

I would be please to be mentor on Hoya.

Regards
JB

On 01/08/2014 04:08 PM, Steve Loughran wrote:
> I'm starting to put together the incubation proposal for Hoya: a tool to
> dynamically deploy applications such as HBase or Accumulo on YARN
>
> https://wiki.apache.org/incubator/HoyaProposal
>
> It does already work to the extent that it can bring up either application,
> run different clusters of different versions, and remember where containers
> were allocated so that on application restart it can ask for them back.
> That increases data locality and makes a big difference with HBase.
>
> It also needs a lot more work -YARN-896 is adding YARN features that help,
> but there's lots of fun to be had in Hoya including
>   -leading edge work in failure handling, modelling cluster unreliability
> and reacting to it. Can we move beyond simple blacklisting to
> "greylisting", accepting unreliable boxes if we have no altenatives
>
> Then there's adding more providers, to support different application
> installations -I'm starting to write a functional test framework which need
> provider-specific workload generations
>
> Other features: AM should have a web ui that redirects to the live
> endpoints to all the app-specific UIs (e.g. HBase Master GUI), as well as
> displaying cluster state itself, for people and for management tools....
>
> To summarise: lots of fun to be had
>
> -steve
>

-- 
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message