incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karthik Kambatla <>
Subject Re: S4 & YARN
Date Wed, 13 Jun 2012 05:47:28 GMT
Hi Arun

It is indeed an exciting proposition to run S4 on Yarn. I believe the delay
to work on S4-25 was mostly due to our focus to get our s4-piper release
out. Now that the release is in the pipeline, it might be a good time to
start thinking about possible integration with Yarn.

I am attending the summit, and would be excited to discuss the challenges
and possibilities of such an integration.


From: Arun C Murthy <>

> Date: Tue, Jun 12, 2012 at 8:26 PM
> Subject: S4 & YARN
> To:
> Folks,
>  I'd like to start a discussion around getting S4 to run within Hadoop
> YARN in hadoop-2.
>  Brief background: Hadoop YARN is a generic resource management framework
> which aims to host multiple applications such as MapReduce, MPI etc. It
> would be very beneficial to get S4 running within YARN since it would make
> it much more accessible for Hadoop users to do real-time processing with S4
> in their existing clusters with almost no operation overheads - thus,
> aiding S4 adoption.
>  I had a brief discussion with Flavio on this topic recently and he
> encouraged me to start a discussion on this list.
>  I see that is already open
> for running S4 within YARN, but hasn't been any activity since.
>  Some more details on YARN and writing applications:
>  I'm happy to help anyone interested in working on this.
> thanks,
> Arun

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message