incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karthik Kambatla <ka...@cloudera.com>
Subject Re: S4 & YARN
Date Wed, 13 Jun 2012 05:47:28 GMT
Hi Arun

It is indeed an exciting proposition to run S4 on Yarn. I believe the delay
to work on S4-25 was mostly due to our focus to get our s4-piper release
out. Now that the release is in the pipeline, it might be a good time to
start thinking about possible integration with Yarn.

I am attending the summit, and would be excited to discuss the challenges
and possibilities of such an integration.

Thanks
Karthik

From: Arun C Murthy <acm@hortonworks.com>

> Date: Tue, Jun 12, 2012 at 8:26 PM
> Subject: S4 & YARN
> To: s4-dev@incubator.apache.org
>
>
> Folks,
>
>  I'd like to start a discussion around getting S4 to run within Hadoop
> YARN in hadoop-2.
>
>  Brief background: Hadoop YARN is a generic resource management framework
> which aims to host multiple applications such as MapReduce, MPI etc. It
> would be very beneficial to get S4 running within YARN since it would make
> it much more accessible for Hadoop users to do real-time processing with S4
> in their existing clusters with almost no operation overheads - thus,
> aiding S4 adoption.
>
>  I had a brief discussion with Flavio on this topic recently and he
> encouraged me to start a discussion on this list.
>
>  I see that https://issues.apache.org/jira/browse/S4-25 is already open
> for running S4 within YARN, but hasn't been any activity since.
>
>  Some more details on YARN and writing applications:
>
> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html
>
>  I'm happy to help anyone interested in working on this.
>
> thanks,
> Arun
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message