helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kishore g <g.kish...@gmail.com>
Subject helix provisioning
Date Tue, 14 Jan 2014 19:27:22 GMT
Hi,

Here is a prototype of supporting the concept of provisioning in Helix.
This was the work done by one of the interns at LinkedIn. You can find more
info here
http://engineering.linkedin.com/cluster-management/auto-scaling-apache-helix-and-apache-yarn

The implementation done by intern was outside of Helix. Kanak and I took a
stab at it to treat provisioning as a first class citizen of Helix.

The work is in helix-provisioning branch

https://git-wip-us.apache.org/repos/asf?p=helix.git;a=tree;h=refs/heads/helix-provisioning;hb=helix-provisioning

Most of the code is here,

https://git-wip-us.apache.org/repos/asf?p=helix.git;a=tree;f=helix-core/src/main/java/org/apache/helix/controller/provisioner;h=2a2650b053206ac4c521bf71899dc286cd0ef961;hb=helix-provisioning

Here is a YARN specific implementation

https://git-wip-us.apache.org/repos/asf?p=helix.git;a=tree;f=helix-provisioning/src/main/java/org/apache/helix/provisioning/yarn;h=93d08917feb88e06140b506eff8f498fcb588fca;hb=helix-provisioning

Its still a WIP and main goal was to see if we can abstract Resource
Managers like YARN, Mesos, EC2 from the application.

I am working on providing the steps to try out launching a Helix cluster on
YARN.

Feedback/Thoughts? Any interest in contributing to make this production
ready ?

thanks,
Kishore G

Mime
View raw message