airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Siddharth Jain <ja...@umail.iu.edu>
Subject Summary: Job Monitor Discussion
Date Sat, 19 Mar 2016 01:06:15 GMT
Hello all,

A discussion was held on 3/17/2016 regarding the implementation of a
separate demon service which reads job status from email and sends it out
to a message bus. For more background information about this topic, please
refer: https://issues.apache.org/jira/browse/AIRAVATA-1912.



Following were the contributors to this discussion:

·       Marlon Pierce

·       Shameera Rathnayaka

·       Siddharth Jain

·       Suresh Marru



Summary of discussion,



*Approaches for receiving messages from job executor/HPC*

1.)    Develop a custom IMAP/SMTP server, which can receive the messages
directly sent by the job executor and place the email on message bus
immediately.

2.)    Setup a simple IMAP/SMTP server on our network, fetch the messages
from it and place the message on message bus.

3.)    Use the third-party provider for email-service, connect to the email
service provider periodically, fetch the messages and put it on the message
bus.

Approach 3 was finalized, because:

·       Maintenance of  custom server or a separate server might be a
challenge in future

·       Third-party email service provider take care of the up-time and any
issues with regular email service



*Approaches for putting messages on message bus*

1.)    Fetch the unread messages from the mail-box and broadcast them as it
is to all the consumer queues bound to exchange. With minimum-invasive
technique have the messages handled with the existing code on the consumer
side.

2.)    Fetch the unread messages, classify them and accordingly route the
messages to appropriate consumer queue. For classifying and determining
which message is sent to which consumer queue, existing logic from GFAC
module can be reused.

3.)    Fetch the unread messages, send all of them to a common endpoint,
which in turn will classify them and accordingly route the messages to
appropriate consumer queues.

Initially approach 1 will be implemented, but the goal is to implement
approach 3 ultimately.



In addition to the above, in general the design has to be such that it can
accommodate consuming messages from different e-mail service providers and
any other protocol (for example UDP)


Please feel free to add anything that has been missed out.


Best regards,
Siddharth Jain

Mime
View raw message