incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthieu Morel <mmo...@apache.org>
Subject Re: How to install S4 on cluster of computers?
Date Wed, 12 Jun 2013 15:02:17 GMT
Hi,

The integration Arun pointed to is for S4 0.5 and YARN 2.0.2-alpha.
This contribution has not yet been ported to S4 0.6 or new version of YARN.

Changes in S4 were mainly: 
- adding an Application Master specific for S4, 
- implementing a mechanism to fetch application code in S4 node from HDFS.

Also note that by default application deployment in S4 is dynamic, i.e. the YARN integration
is only for starting an S4 cluster, app deployment is somehow independent (app code is fetched
from a specified location).


Packaging/deploying with BigTop sounds very appealing, it would provide the ability to easily
integrate a stream processing capability to a hadoop stack. It's hard for me to volunteer
at the moment for that, but I would be glad to help anyone willing to work on it.

Matthieu




On Jun 11, 2013, at 02:01 , Roman Shaposhnik <rvs@apache.org> wrote:

> On Mon, Jun 10, 2013 at 4:31 PM, Arun C Murthy <acm@hortonworks.com> wrote:
>> https://cwiki.apache.org/S4/deploying-s4-applications-with-yarn.html is another option.
>> 
>> I'll let Flavio and others comment on where it stands ATM.
> 
> Great point! Didn't know it existed. This makes me want to see S4 in Bigtop
> even more -- would be a great test for a more of a framework aspect of YARN.
> 
> The best we've done so far is Giraph on YARN in Bigtop.
> 
> Thanks,
> Roman.


Mime
View raw message