incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Junqueira <...@yahoo-inc.com>
Subject Re: S4 & YARN
Date Wed, 13 Jun 2012 10:26:18 GMT
I'm interested in this integration, but I need to wrap my head around the deployment model
of piper first. 

-Flavio 

On Jun 13, 2012, at 5:26 AM, Arun C Murthy wrote:

> Folks,
> 
> I'd like to start a discussion around getting S4 to run within Hadoop YARN in hadoop-2.
> 
> Brief background: Hadoop YARN is a generic resource management framework which aims to
host multiple applications such as MapReduce, MPI etc. It would be very beneficial to get
S4 running within YARN since it would make it much more accessible for Hadoop users to do
real-time processing with S4 in their existing clusters with almost no operation overheads
- thus, aiding S4 adoption.
> 
> I had a brief discussion with Flavio on this topic recently and he encouraged me to start
a discussion on this list.
> 
> I see that https://issues.apache.org/jira/browse/S4-25 is already open for running S4
within YARN, but hasn't been any activity since.
> 
> Some more details on YARN and writing applications:
> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html
> 
> I'm happy to help anyone interested in working on this.
> 
> thanks,
> Arun
> 

flavio
junqueira
senior research scientist
 
fpj@yahoo-inc.com
direct +34 93-183-8828
 
avinguda diagonal 177, 8th floor, barcelona, 08018, es
phone (408) 349 3300    fax (408) 349 3301


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message