helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kanak Biscuitwala <kana...@hotmail.com>
Subject RE: helix provisioning
Date Sat, 18 Jan 2014 17:36:20 GMT
Maybe it should be at both scopes? Sort of like putting bounds cross-app and cross-service
within an app. This allows full control over an organization's physical resources.

> Date: Sat, 18 Jan 2014 09:28:32 -0800
> Subject: Re: helix provisioning
> From: g.kishore@gmail.com
> To: dev@helix.apache.org; user@helix.apache.org
> 
> Thanks Swaroop for adding interfaces to specify the application spec.
> 
> https://git-wip-us.apache.org/repos/asf?p=helix.git;a=blob;f=helix-provisioning/src/main/java/org/apache/helix/provisioning/yarn/ApplicationSpec.java;h=16b23fa02ffbe40c18b9cff286eb5eb321947d84;hb=helix-provisioning
> 
> From what I understand
> AppName maps to cluster name
> Service maps to instances, there can be multiple service components in an
> application. For example, in distributed data system, there may be a
> storage layer and router layer.
> 
> The min/max containers seems to be at application level but it probably
> should be at a Service scope ?
> 
> what do you think ?
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On Tue, Jan 14, 2014 at 11:27 AM, kishore g <g.kishore@gmail.com> wrote:
> 
> > Hi,
> >
> > Here is a prototype of supporting the concept of provisioning in Helix.
> > This was the work done by one of the interns at LinkedIn. You can find more
> > info here
> >
> > http://engineering.linkedin.com/cluster-management/auto-scaling-apache-helix-and-apache-yarn
> >
> > The implementation done by intern was outside of Helix. Kanak and I took a
> > stab at it to treat provisioning as a first class citizen of Helix.
> >
> > The work is in helix-provisioning branch
> >
> >
> > https://git-wip-us.apache.org/repos/asf?p=helix.git;a=tree;h=refs/heads/helix-provisioning;hb=helix-provisioning
> >
> > Most of the code is here,
> >
> >
> > https://git-wip-us.apache.org/repos/asf?p=helix.git;a=tree;f=helix-core/src/main/java/org/apache/helix/controller/provisioner;h=2a2650b053206ac4c521bf71899dc286cd0ef961;hb=helix-provisioning
> >
> > Here is a YARN specific implementation
> >
> >
> > https://git-wip-us.apache.org/repos/asf?p=helix.git;a=tree;f=helix-provisioning/src/main/java/org/apache/helix/provisioning/yarn;h=93d08917feb88e06140b506eff8f498fcb588fca;hb=helix-provisioning
> >
> > Its still a WIP and main goal was to see if we can abstract Resource
> > Managers like YARN, Mesos, EC2 from the application.
> >
> > I am working on providing the steps to try out launching a Helix cluster
> > on YARN.
> >
> > Feedback/Thoughts? Any interest in contributing to make this production
> > ready ?
> >
> > thanks,
> > Kishore G
> >
> >
> >
> >
> >
> >
> >
> >
 		 	   		  
Mime
View raw message