airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Siddharth Jain <>
Subject Re: [GSoC Proposal] Apache Airavata Monitoring Module
Date Fri, 25 Mar 2016 09:30:43 GMT
Hi Supun,
Thank you for the pointers :)

I have included the high level working/architecture diagram in the final

The extension part, what I really meant in there is, the architecture will
be such that, new source/channel of job status updates could be
accommodated in future easily. This actually came up during the discussion
I had with Marlon, Shameera and Suresh. Suresh was like, though right now
we are reading the job status updates from email, there is a possibility in
future we might be getting these updates from some other source, like UDP,
so the architecture of the module I propose to design should be able
extensible, like plug and play. Its like something what we anticipate. So I
have kept that in mind while coming up with the architecture of module,
basically kept each component cohesive. Let me explain it to you with a

‚ÄčThe top layer "raw message endpoint" only deals with raw messages. Right
now what we plan to have is just an email fetcher in there, which will
fetch e-mails from various mailboxes. But say we have to support UDP
message tomorrow, what needs to be done is, we need to implement code to
listen to a specific UDP port and collect the raw message in that layer.
Now once we have the raw message, we will pass it  to Job Status Extractor
layer. Job Status Extractor layer will really be like an interface which
has the method extract(), which parses the given raw message (according to
the specific raw message type, so like bunch of extractors for each type of
channel in this layer) and extracts the job status information object. So
an extractor for the specific UDP message needs to be implemented and wired
in the layer. The trick to keep everything underneath this layer unaffected
is that we come up with a common contract for job status information
object, i.e. the basic attributes we need for representing job status
information will always remain same. So the identfier and publisher layers
will be unaffected. I hope this makes sense.

Again, appreciate you taking time out to review my proposal.

Siddharth Jain

On Wed, Mar 23, 2016 at 1:37 PM, Supun Nakandala <>

> Hi Siddharth,
> Can you provide an architecture diagram for the proposed solution. That
> will help us to understand how the monitoring module will work. Also you
> have said that the module can be extended to support more types of
> monitoring types. Can you explain more on this idea. Also if you can
> provide an implementation of another monitoring type, it will be a strong
> evidence on the generalizability of the module.
> Some of the existing challenges that we have related to monitoring are
>   1. monitoring jobs running on a mesos cluster
>   2. monitoring jobs running on a local node
>   3. monitoring jobs running on a remote node but does not have any queue
> manager like slurm or pbs
> On Wed, Mar 23, 2016 at 12:52 AM, Siddharth Jain <>
> wrote:
>> Correction, the link is:
>> <>
>> On Tue, Mar 22, 2016 at 6:35 PM, Siddharth Jain <>
>> wrote:
>>> Hello all,
>>> I have the first draft of my GSoC proposal on Apache Airavata Monitoring
>>> Module ready. I will appreciate if you could give any comments or
>>> suggestions to improve this proposal.
>>> The proposal is available on:
>>> i)
>>> ii)
>>> The link points to what it will actually look like,
>>> the google docs version is just for convenience of commenting.
>>> Best regards,
>>> Siddharth Jain
> --
> Thank you
> Supun Nakandala
> Dept. Computer Science and Engineering
> University of Moratuwa

View raw message