incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gowtham S <seldomgowt...@gmail.com>
Subject Re: Problem in running a same PE in multiple nodes
Date Fri, 22 Feb 2013 13:02:26 GMT
sorry for the wrong thread and gimp format..

here i have attached the .png format of the screenshot which shows that all
the 4 nodes are active .

and also i didnt modify the hash function. when ever i run it in 4 nodes,
it always map the 4 different keyed PE to four different nodes. i didnt
manually  change the hash function ..



On Thu, Feb 21, 2013 at 1:30 AM, Matthieu Morel <mmorel@apache.org> wrote:

> Hi Jihyoun,
>
> PE graphs with cycles can certainly get a bit tricky sometimes, but mainly
> because they can lead to deadlocks.
>
> I don't see how you'd lose events in the case you present if you don't
> lose or reconfigure nodes. In the very specific case of sending an event
> from PE instance a1 to the same PE instance a1, the event is simply put in
> an in-memory blocking queue.
>
> It would be great to understand in which conditions you can see this
> issue. Could you be more specific? (I can't reproduce that with simple
> tests, but I don't have your own settings/code/environment).
>
> Thanks,
>
> Matthieu
>
> On Feb 20, 2013, at 03:04 , JiHyoun Park wrote:
>
> Dear Matthieu
>
> I also experienced the same problem with Gowtham.
> Unlike him, I created a downstream to itself with only one key. (It was
> like a flag to be turned on/off.)
> I tested it and found sometimes one PE instance does not receive the
> event.
> I remember that it didn't depend on a specific node.
>
> At the time, I also wondered why.
> Since it's a very simple logic, you can test it by yourself without much
> efforts.
> It would be really appreciated if you can tell us the solution or at least
> the reason.
>
> Best Regards
> Jihyoun
>
>
>
> On Tue, Feb 19, 2013 at 11:38 PM, Matthieu Morel <mmorel@apache.org>wrote:
>
>> Hi,
>>
>> Can you provide more detail about a node not running correctly? what does
>> that mean to you, that it does not receive messages? Maybe that's normal
>> because of the distribution of keys. Or maybe you only have 3 active nodes
>> in the cluster. (you can check that with the s4 status tool)
>>
>> The distribution of keys across nodes is a simple hash+mod , which you
>> can override if needed.
>>
>> I would recommend to check whether all data is processed as expected,
>> regardless of the location.
>>
>> Let us know,
>>
>> thanks,
>>
>> Matthieu
>>
>>
>> On Feb 19, 2013, at 16:27 , Gowtham S wrote:
>>
>> Hi S4 community,
>>
>> I have been working with s4 piper for 3 months. now i am just trying to
>> make a downstream from a PE to the same PE with different Key Id.
>>
>> i just have a scenario in which i would have an unique PE whose
>> downstream is also to that PE  but each time the Key Id of the targeted PE
>> may vary. this continues until it satisfies a condition where it just stops.
>>
>> i started implementing this in a single machine with 4 nodes, as i had 4
>> different keys for that unique PE(say 0,1,2,3). so now every node will have
>> that PE clone with one individual key Id from the available Key Ids. one PE
>> may send some events to PE in other  nodes , until certain condition is
>> reached where the process is stopped.
>>
>> now three nodes are running correctly except one node.. i wonder why ..
>> because all the nodes have same computational part to execute.
>>
>> please help me as how s4 deals while assigning a single PE to different
>> nodes with different key Id ..
>>
>>
>>
>>
>>
>>
>> On Tue, Feb 19, 2013 at 8:22 PM, Gowtham S <seldomgowtham@gmail.com>wrote:
>>
>>> Hi S4 community,
>>>
>>>
>>
>>
>
>

Mime
View raw message