airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pierce, Marlon" <marpi...@iu.edu>
Subject Re: Summary: Job Monitor Discussion
Date Sat, 19 Mar 2016 01:12:36 GMT
Nice summary, Siddharth.

From: Siddharth Jain <jain8@umail.iu.edu<mailto:jain8@umail.iu.edu>>
Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" <dev@airavata.apache.org<mailto:dev@airavata.apache.org>>
Date: Friday, March 18, 2016 at 9:06 PM
To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" <dev@airavata.apache.org<mailto:dev@airavata.apache.org>>
Subject: Summary: Job Monitor Discussion

Hello all,
A discussion was held on 3/17/2016 regarding the implementation of a separate demon service
which reads job status from email and sends it out to a message bus. For more background information
about this topic, please refer: https://issues.apache.org/jira/browse/AIRAVATA-1912.

Following were the contributors to this discussion:

·       Marlon Pierce

·       Shameera Rathnayaka

·       Siddharth Jain

·       Suresh Marru


Summary of discussion,

Approaches for receiving messages from job executor/HPC

1.)    Develop a custom IMAP/SMTP server, which can receive the messages directly sent by
the job executor and place the email on message bus immediately.

2.)    Setup a simple IMAP/SMTP server on our network, fetch the messages from it and place
the message on message bus.

3.)    Use the third-party provider for email-service, connect to the email service provider
periodically, fetch the messages and put it on the message bus.
Approach 3 was finalized, because:

·       Maintenance of  custom server or a separate server might be a challenge in future

·       Third-party email service provider take care of the up-time and any issues with regular
email service


Approaches for putting messages on message bus

1.)    Fetch the unread messages from the mail-box and broadcast them as it is to all the
consumer queues bound to exchange. With minimum-invasive technique have the messages handled
with the existing code on the consumer side.

2.)    Fetch the unread messages, classify them and accordingly route the messages to appropriate
consumer queue. For classifying and determining which message is sent to which consumer queue,
existing logic from GFAC module can be reused.

3.)    Fetch the unread messages, send all of them to a common endpoint, which in turn will
classify them and accordingly route the messages to appropriate consumer queues.
Initially approach 1 will be implemented, but the goal is to implement approach 3 ultimately.

In addition to the above, in general the design has to be such that it can accommodate consuming
messages from different e-mail service providers and any other protocol (for example UDP)

Please feel free to add anything that has been missed out.

Best regards,
Siddharth Jain
Mime
View raw message