storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject Message Passing Implementation
Date Sun, 08 Nov 2020 17:36:47 GMT

I hope my email finds all of you well. I am a Phd candidate and I am 
writing to you with regard to scheduling in Apache Storm. According to 
the MPI implementation of Storm the workers manage connections to other 
workers and maintain a mapping from task to task. Also, transferring 
takes in a task id and a tuple, and it serializes the tuple and puts it 
onto a "transfer queue”.

The question is, if there is a way to organise scheduling, such that 
certain tasks of an operator communicate to only certain tasks of the 
following operator at a given time according to the application’s 
topology (could ZeroMQ possibly do something like this?). Any hint would 

Thank you in advance for your valuable help.

Kind regards,

View raw message