incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthieu Morel <mmo...@apache.org>
Subject Re: Problem in running a same PE in multiple nodes
Date Wed, 20 Feb 2013 20:00:53 GMT
Hi Jihyoun,

PE graphs with cycles can certainly get a bit tricky sometimes, but mainly because they can
lead to deadlocks.

I don't see how you'd lose events in the case you present if you don't lose or reconfigure
nodes. In the very specific case of sending an event from PE instance a1 to the same PE instance
a1, the event is simply put in an in-memory blocking queue.

It would be great to understand in which conditions you can see this issue. Could you be more
specific? (I can't reproduce that with simple tests, but I don't have your own settings/code/environment).

Thanks,

Matthieu

On Feb 20, 2013, at 03:04 , JiHyoun Park wrote:

> Dear Matthieu
> 
> I also experienced the same problem with Gowtham.
> Unlike him, I created a downstream to itself with only one key. (It was like a flag to
be turned on/off.)
> I tested it and found sometimes one PE instance does not receive the event. 
> I remember that it didn't depend on a specific node.
> 
> At the time, I also wondered why.
> Since it's a very simple logic, you can test it by yourself without much efforts.
> It would be really appreciated if you can tell us the solution or at least the reason.
> 
> Best Regards
> Jihyoun
> 
> 
> 
> On Tue, Feb 19, 2013 at 11:38 PM, Matthieu Morel <mmorel@apache.org> wrote:
> Hi,
> 
> Can you provide more detail about a node not running correctly? what does that mean to
you, that it does not receive messages? Maybe that's normal because of the distribution of
keys. Or maybe you only have 3 active nodes in the cluster. (you can check that with the s4
status tool)
> 
> The distribution of keys across nodes is a simple hash+mod , which you can override if
needed.
> 
> I would recommend to check whether all data is processed as expected, regardless of the
location.
> 
> Let us know,
> 
> thanks,
> 
> Matthieu
> 
> 
> On Feb 19, 2013, at 16:27 , Gowtham S wrote:
> 
>> Hi S4 community,
>> 
>> I have been working with s4 piper for 3 months. now i am just trying to make a downstream
from a PE to the same PE with different Key Id. 
>> 
>> i just have a scenario in which i would have an unique PE whose downstream is also
to that PE  but each time the Key Id of the targeted PE may vary. this continues until it
satisfies a condition where it just stops.
>> 
>> i started implementing this in a single machine with 4 nodes, as i had 4 different
keys for that unique PE(say 0,1,2,3). so now every node will have that PE clone with one individual
key Id from the available Key Ids. one PE may send some events to PE in other  nodes , until
certain condition is reached where the process is stopped.
>> 
>> now three nodes are running correctly except one node.. i wonder why .. because all
the nodes have same computational part to execute. 
>> 
>> please help me as how s4 deals while assigning a single PE to different nodes with
different key Id ..
>> 
>> 
>> 
>>  
>> 
>> 
>> On Tue, Feb 19, 2013 at 8:22 PM, Gowtham S <seldomgowtham@gmail.com> wrote:
>> Hi S4 community,
>> 
>> 
> 
> 


Mime
View raw message