helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kishore g <g.kish...@gmail.com>
Subject Re: helix provisioning
Date Sat, 18 Jan 2014 17:28:32 GMT
Thanks Swaroop for adding interfaces to specify the application spec.

https://git-wip-us.apache.org/repos/asf?p=helix.git;a=blob;f=helix-provisioning/src/main/java/org/apache/helix/provisioning/yarn/ApplicationSpec.java;h=16b23fa02ffbe40c18b9cff286eb5eb321947d84;hb=helix-provisioning

>From what I understand
AppName maps to cluster name
Service maps to instances, there can be multiple service components in an
application. For example, in distributed data system, there may be a
storage layer and router layer.

The min/max containers seems to be at application level but it probably
should be at a Service scope ?

what do you think ?










On Tue, Jan 14, 2014 at 11:27 AM, kishore g <g.kishore@gmail.com> wrote:

> Hi,
>
> Here is a prototype of supporting the concept of provisioning in Helix.
> This was the work done by one of the interns at LinkedIn. You can find more
> info here
>
> http://engineering.linkedin.com/cluster-management/auto-scaling-apache-helix-and-apache-yarn
>
> The implementation done by intern was outside of Helix. Kanak and I took a
> stab at it to treat provisioning as a first class citizen of Helix.
>
> The work is in helix-provisioning branch
>
>
> https://git-wip-us.apache.org/repos/asf?p=helix.git;a=tree;h=refs/heads/helix-provisioning;hb=helix-provisioning
>
> Most of the code is here,
>
>
> https://git-wip-us.apache.org/repos/asf?p=helix.git;a=tree;f=helix-core/src/main/java/org/apache/helix/controller/provisioner;h=2a2650b053206ac4c521bf71899dc286cd0ef961;hb=helix-provisioning
>
> Here is a YARN specific implementation
>
>
> https://git-wip-us.apache.org/repos/asf?p=helix.git;a=tree;f=helix-provisioning/src/main/java/org/apache/helix/provisioning/yarn;h=93d08917feb88e06140b506eff8f498fcb588fca;hb=helix-provisioning
>
> Its still a WIP and main goal was to see if we can abstract Resource
> Managers like YARN, Mesos, EC2 from the application.
>
> I am working on providing the steps to try out launching a Helix cluster
> on YARN.
>
> Feedback/Thoughts? Any interest in contributing to make this production
> ready ?
>
> thanks,
> Kishore G
>
>
>
>
>
>
>
>

Mime
View raw message