airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shenoy, Gourav Ganesh" <>
Subject [#Spring17-Airavata-Courses] : Distributed Workload Management for Airavata
Date Wed, 01 Feb 2017 17:52:52 GMT
Hello dev, arch,

As part of this Spring’17 Advanced Science Gateway Architecture course, we are working on
trying to debate and find possible solutions to the issue of managing distributed workloads
in Apache Airavata. This leads to the discussion of finding the most efficient way that different
Airavata micro-services should communicate and distribute work, in such a way that:

1.       We maintain the ability to scale these micro-services whenever needed (autoscale

2.       Achieve fault tolerance.

3.       We can deploy these micro-services independently, or better in a containerized manner
– keeping in mind the ability to use devops for deployment.

As of now the options we are exploring are:

1.       RPC based communication

2.       Message based – either master-worker, or work-queue, etc

3.       A combination of both these approaches

I am more inclined towards exploring the message based approach, but again there arises the
possibility of handling limitations/corner cases of message broker such as downtimes (may
be more). In my opinion, having asynchronous communication will help us achieve most of the
above-mentioned points. Another debatable issue is making the micro-services implementation
stateless, such that we do not have to pass the state information between micro-services.

I would love to hear any thoughts/suggestions/comments on this topic and open up a discussion
via this mail thread. If there is anything that I have missed which is relevant to this issue,
please let me know.

Thanks and Regards,
Gourav Shenoy
View raw message