Return-Path: X-Original-To: apmail-airavata-dev-archive@www.apache.org Delivered-To: apmail-airavata-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B2E09187B7 for ; Sat, 19 Mar 2016 01:06:18 +0000 (UTC) Received: (qmail 27155 invoked by uid 500); 19 Mar 2016 01:06:18 -0000 Delivered-To: apmail-airavata-dev-archive@airavata.apache.org Received: (qmail 27107 invoked by uid 500); 19 Mar 2016 01:06:18 -0000 Mailing-List: contact dev-help@airavata.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airavata.apache.org Delivered-To: mailing list dev@airavata.apache.org Received: (qmail 27097 invoked by uid 99); 19 Mar 2016 01:06:17 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 19 Mar 2016 01:06:17 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 71700C24C0 for ; Sat, 19 Mar 2016 01:06:17 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.281 X-Spam-Level: * X-Spam-Status: No, score=1.281 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, HTML_OBFUSCATE_05_10=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=umail-iu-edu.20150623.gappssmtp.com Received: from mx2-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id qAvkf-RRxMOp for ; Sat, 19 Mar 2016 01:06:16 +0000 (UTC) Received: from mail-vk0-f52.google.com (mail-vk0-f52.google.com [209.85.213.52]) by mx2-lw-us.apache.org (ASF Mail Server at mx2-lw-us.apache.org) with ESMTPS id 03F355F56E for ; Sat, 19 Mar 2016 01:06:15 +0000 (UTC) Received: by mail-vk0-f52.google.com with SMTP id e6so161344459vkh.2 for ; Fri, 18 Mar 2016 18:06:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=umail-iu-edu.20150623.gappssmtp.com; s=20150623; h=mime-version:date:message-id:subject:from:to; bh=9DnPQtc6+WWsynpfVb0jXGiTp+mxKWuPrh3DWYiKPX0=; b=oWa7ckpkp4F2r6MYFezqwFV1ej5ytdxdfpPdGAcxID3f+Huz4L7HcTWTsVuKWjREsp 4gAndy8EbzVe5zviykkQdBbIlKryZ9b7DeAggxNg8hp9WpOkjVQc88eGvy18IpKRl8Xh LJzrtHALbaojNqnYorPq+F7RL9V+BYLXhoil8P0wUT6+TKU7gDmMuhblso8GKcCz4iOR i2ge7kGBAyEsKlV5htas4jtkDEg9ryB6oMpoeqBDy0j+ewPfiYkErYH6Ca8+vfgCfN4L SmPT58JKT+B72NcWAGehQ8yNNOock+WoU0cRIb68dwH6ytGb7CEdrpXfYQUMv3IXJSAE T3uA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to; bh=9DnPQtc6+WWsynpfVb0jXGiTp+mxKWuPrh3DWYiKPX0=; b=HZ3VSxG1bZVc263ydYsRqapNLkZf/MHHLozc6+gX7dXShIAPItxSfwazFEpMibb3DV ki+Usozaw9nyjO6XYhwNVAJNYg2+TXKsaW2If7R97VAgJmj5IFsFswo9pvmNgMOY2rzO sRG7HIRcaO2s0Jq8l4aGldKzUx7OkUZl4oll4j1STLxULuiK7m/MJUrD9sq30/c0Q/Mw LCzaNX9f31jIWRU21u/lDWduNCBsrob1sH5sYB89uKWjo3JZKMZo23oXu2mueAM5TlPu Qr7ySR3OzZ+6AST9vDGE3JLAoDHbsMtvzOfpJLyy9SFcyiLk6AWRK+DoAqX67/DmmCjW 6LGw== X-Gm-Message-State: AD7BkJLP8RLTO9CWTWY3x6Fftm5gnTOQ03Yrct5w105qlZgOiNFbIbGAlnDc8P+U5xSzt6AnHx2mqjMvXTmsgvFB MIME-Version: 1.0 X-Received: by 10.31.139.1 with SMTP id n1mr20262324vkd.33.1458349575312; Fri, 18 Mar 2016 18:06:15 -0700 (PDT) Received: by 10.176.68.165 with HTTP; Fri, 18 Mar 2016 18:06:15 -0700 (PDT) Date: Fri, 18 Mar 2016 21:06:15 -0400 Message-ID: Subject: Summary: Job Monitor Discussion From: Siddharth Jain To: dev@airavata.apache.org Content-Type: multipart/alternative; boundary=001a1145a02e652607052e5c78e1 --001a1145a02e652607052e5c78e1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hello all, A discussion was held on 3/17/2016 regarding the implementation of a separate demon service which reads job status from email and sends it out to a message bus. For more background information about this topic, please refer: https://issues.apache.org/jira/browse/AIRAVATA-1912. Following were the contributors to this discussion: =C2=B7 Marlon Pierce =C2=B7 Shameera Rathnayaka =C2=B7 Siddharth Jain =C2=B7 Suresh Marru Summary of discussion, *Approaches for receiving messages from job executor/HPC* 1.) Develop a custom IMAP/SMTP server, which can receive the messages directly sent by the job executor and place the email on message bus immediately. 2.) Setup a simple IMAP/SMTP server on our network, fetch the messages from it and place the message on message bus. 3.) Use the third-party provider for email-service, connect to the email service provider periodically, fetch the messages and put it on the message bus. Approach 3 was finalized, because: =C2=B7 Maintenance of custom server or a separate server might be a challenge in future =C2=B7 Third-party email service provider take care of the up-time an= d any issues with regular email service *Approaches for putting messages on message bus* 1.) Fetch the unread messages from the mail-box and broadcast them as it is to all the consumer queues bound to exchange. With minimum-invasive technique have the messages handled with the existing code on the consumer side. 2.) Fetch the unread messages, classify them and accordingly route the messages to appropriate consumer queue. For classifying and determining which message is sent to which consumer queue, existing logic from GFAC module can be reused. 3.) Fetch the unread messages, send all of them to a common endpoint, which in turn will classify them and accordingly route the messages to appropriate consumer queues. Initially approach 1 will be implemented, but the goal is to implement approach 3 ultimately. In addition to the above, in general the design has to be such that it can accommodate consuming messages from different e-mail service providers and any other protocol (for example UDP) Please feel free to add anything that has been missed out. Best regards, Siddharth Jain --001a1145a02e652607052e5c78e1 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Hello all,

A discussion was held on 3/17/2016 regarding the implementation of a separate demon service which reads job status from emai= l and sends it out to a message bus. For more background information about this t= opic, please refer: https://issues.apache.org/jira/browse/AIRAVATA-1912.

=C2=A0

Following were the contributors to this discussion:<= /p>

=C2=B7=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Marlon Pierce

=C2=B7=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Shameera Rathnayaka

=C2=B7=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Siddharth Jain

=C2=B7=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Suresh Marru

=C2=A0

Summary of discussion,

=C2=A0

Approaches = for receiving messages from job executor/HPC

1.)=C2=A0=C2=A0=C2= =A0 Develop a custom IMAP/SMTP server, which can receive the messages directly sent by the job executor and place the email = on message bus immediately.

2.)=C2=A0=C2=A0=C2= =A0 Setup a simple IMAP/SMTP server on our network, fetch the messages from it and place the message on message bus.

3.)=C2=A0=C2=A0=C2= =A0 Use the third-party provider for email-service, connect to the email service provider periodically, fetch the messages and = put it on the message bus.

Approach 3 was finalized, because:

=C2=B7=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Maintenance of=C2=A0 custom server or a separate server might be a challenge in future

=C2=B7=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Third-party email service provider take care of the up-time and any issues with regular email service

=C2=A0

Approaches = for putting messages on message bus

1.)=C2=A0=C2=A0=C2= =A0 Fetch the unread messages from the mail-box and broadcast them as it is to all the consumer queues bound to exchange. With minimum-invasive technique have the messages handled with the existing code= on the consumer side.

2.)=C2=A0=C2=A0=C2= =A0 Fetch the unread messages, classify them and accordingly route the messages to appropriate consumer queue. For classifyi= ng and determining which message is sent to which consumer queue, existing log= ic from GFAC module can be reused.

3.)=C2=A0=C2=A0=C2= =A0 Fetch the unread messages, send all of them to a common endpoint, which in turn will classify them and accordingly route the messages to appropriate consumer queues.

Initially approach 1 will be implemented, but the go= al is to implement approach 3 ultimately.

=C2=A0

In addition to the above, in general the design has = to be such that it can accommodate consuming messages from different e-mail servi= ce providers and any other protocol (for example UDP)

=C2=A0

Please feel fre= e to add anything that has been missed out.

=C2=A0

Best regards,Siddharth Jain
--001a1145a02e652607052e5c78e1--