airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Marru (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (AIRAVATA-964) Add single job support as a first class simple Apache Airavata Execution
Date Tue, 10 Dec 2013 13:19:07 GMT

    [ https://issues.apache.org/jira/browse/AIRAVATA-964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843937#comment-13843937
] 

Suresh Marru edited comment on AIRAVATA-964 at 12/10/13 1:18 PM:
-----------------------------------------------------------------

Lahiru started a discussion of this topic on the dev list - http://markmail.org/thread/yprzdxa2znyfrsl3
Since the discussion covered a big topic, it was going long leading to offline documents.
How about we break it down and discuss one step after another. I will take a stab with first
initial steps.

Referring to the attached diagram. 
Assumption: Airavata API will be extended to include a simplified execution manager. The new
orchestrater component will implement the simple execution API. 

Step A: Orchestrator will accept incoming requests and persist the request as is in Airavata
Registry with a handle (in form of experiment id) returned to the client.
Next the orchestrator will marshall the request, validate user input, verify the required
computational security credentials are accessible, bind (schedule) the required computational
resource and record the translated GFac consumable request in the registry.

Step B: The orchestrator then finds a GFac instance to execute the request and delegates to
process the experiment Id . 

Step C: The GFac instance then queries registry with the experiment id, fetches all required
information and keep updating the registry at well defined check points.

I will stop it here and how about we discuss if the above steps are captured and from the
mailing lists discussion and any debates on any steps? 


was (Author: smarru):
Lahiru started a discussion of this topic on the dev list - http://markmail.org/thread/yprzdxa2znyfrsl3
Since the discussion covered a big topic, it was going long leading to offline documents.
How about we break it down and discuss one step after another. I will take a stab with first
initial steps. Referring to the attached diagram. First lets extend the current Airavata API
to include a simplified execution manager. The new orchestrater component will implement the
simple execution API and take incoming requests in persist the request as is in Airavata Registry
with a handle (in form of experiment id) returned to the client. This is indicated in Step
A in the attached figure. Next the orchestrator will marshall the request, validate user input,
verify the required computational security credentials are accessible, bind (schedule) the
required computational resource and record the translated GFac consumable request in the registry.
The orchestrator then finds a GFac instance to execute the request and invokes it with the
experiment Id as indicated in Step B. The Gfac instance then queries registry with the experiment
id, fetches all required information and keep updating the registry at well defined check
points.

I will stop it here and how about we discuss if the above steps are captured and from the
mailing lists discussion and any debates on any steps? 


> Add single job support as a first class simple Apache Airavata Execution
> ------------------------------------------------------------------------
>
>                 Key: AIRAVATA-964
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-964
>             Project: Airavata
>          Issue Type: Epic
>          Components: Airavata Client, GFac
>            Reporter: Suresh Marru
>              Labels: Architecture
>         Attachments: Airavata-Single-Job-Execution.png
>
>
> Currently Airavata supports single jobs by wrapping it as a single node task within a
workflow. This wrapping is justified if workflow is the predominant use of Airavata and single
application execution is a rare usage pattern. As Airavata is getting more usage, the simple
execution but for large number of invocations and diverse applications is getting more widely
used. 
> Providing a first class way of a simple application execution will reduce the overhead
in creating and managing workflows. But this takes away all the orchestration capabilities
provided by the Workflow Interpreter. This Epic is to discuss a new component to Airavata
which provides based job orchestration while persisting the request state to registry so the
application management component (GFac) can be stateless and recover the job from a frequently
checkpointed state from the registry



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Mime
View raw message