incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Junqueira <...@s4.io>
Subject Re: [PROPOSAL] S4 for the Apache Incubator
Date Mon, 19 Sep 2011 17:13:01 GMT
Hi Vinod, Thanks for the support and comments. See my more comments  
below:

On Sep 18, 2011, at 6:55 AM, Vinod Kumar Vavilapalli wrote:

> Great addition to Apache, +1.
>
> Some comments on the proposal:
> - In the "Initial Goals" section, you referred to "Add a cluster  
> management
> system". Not sure of what it entirely means, but if it is what I  
> think it
> is, the new resource-management layer in Hadoop (YARN/ JIRA issue
> MAPREDUCE-279) can be leveraged for this. YARN is going to be  
> available in
> Hadoop 0.23 release. Willing to help in this regard in any way  
> possible if
> you think it makes sense for S4, can have a separate discussion too.

To give you an idea of what we have in mind, here are a few points:

- Monitor Nodes and PEs.
- Ability to  query specific PE instance variables.
- Add remove nodes.
- Monitor logs.
- Monitor CPU/Memory usage per node.
- Load/unload apps.

In general, it sounds right to assess if it is viable to use YARN for  
S4 cluster management and we would appreciate your help. If it is ok  
with you, I'd rather not include it in the proposal, though, since it  
is an issue we could consider independently of the project going into  
incubator. Let me know if you agree.

> - You may want to capture the discussion about Hdaoop/Flume/Kafka in  
> the
> "Relationships with Other Apache Projects" section.

I have added a summary to the proposal, thanks for pointing it out.

> - Seen elsewhere that proposals list down the list of companies of the
> initial set of committers (section "Homogeneous Developers").
>

I have also added a list here.

-Flavio

>
> On Thu, Sep 15, 2011 at 1:49 AM, Leo Neumeyer  
> <leoneumeyer@gmail.com> wrote:
>
>> Dear all,
>>
>> I would like to propose S4 to be an Apache Incubator project.  S4  
>> is a
>> distributed streaming platform written in Java.
>>
>> Here is a link to the proposal in the Incubator wiki:
>> http://wiki.apache.org/incubator/S4Proposal
>>
>> Thanks,
>> Leo Neumeyer
>>
>> http://s4.io
>> http://twitter.com/leoneu
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message