Mailing-List: contact dev-help@airavata.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airavata.apache.org
Received-SPF: pass (athena.apache.org: domain of samindaw@gmail.com designates
 209.85.215.45 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAB494cwwRwzwKX-+CNrQG65=308QKEN=UZ_hRCP+cf44pdLLiA@mail.gmail.com>
References: 
 <CAB494cwK9mGYXWw6Orf939KeHT=OEJweL4iHfrFrOmmAe+6e9w@mail.gmail.com>
 <CANotcz4my9EXBz0ZUK_H=_sgdFCwUe+TZbe2jj1UoDVXbb7wrA@mail.gmail.com>
 <CAB494cwwRwzwKX-+CNrQG65=308QKEN=UZ_hRCP+cf44pdLLiA@mail.gmail.com>
From: Saminda Wijeratne <samindaw@gmail.com>
Date: Tue, 5 Feb 2013 17:17:02 -0500
Message-ID: 
 <CANotcz4K28-D67mSRobAkc1ig+hAOdM3bZ8o1jKwd1M9-XEUow@mail.gmail.com>
Subject: Re: API Client questions
To: dev@airavata.apache.org
Content-Type: multipart/alternative; boundary=e0cb4efe2f12dfe5d904d5019205

--e0cb4efe2f12dfe5d904d5019205
Content-Type: text/plain; charset=ISO-8859-1

On Tue, Feb 5, 2013 at 4:46 PM, Pedro da Silveira <pedrorcs@gmail.com>wrote:

> Saminda,
>
>
> On Tue, Feb 5, 2013 at 2:18 PM, Saminda Wijeratne <samindaw@gmail.com
> >wrote:
>
> > Hi Pedro,
> >
> > see my comments in-line,
> >
> > On Tue, Feb 5, 2013 at 1:42 PM, Pedro da Silveira <pedrorcs@gmail.com
> > >wrote:
> >
> > > Hi Dev,
> >> >
> >> > I have a few question regarding to Airavata API Client.
> >> > I tested both 2 samples from the directory
> >> > "apache-airavata-client-0.7-SNAPSHOT/samples". They all ran the echo
> >> > example successfully.
> >> > I modified the sample "workflow-run" to submit my pre-defined
> workflow,
> >> > which has 4 inputs file "URI", 2 Application Services and total wall
> >> time
> >> > 20 minutes. My workflow ran successfully using API Client as it did on
> >> > Xbaya.
> >> >
> >> > 1 - One part of the code that got my attention was a call to the
> method
> >> > "getWorkflowComposeContent()", which reads my workflow saved file with
> >> > extension ".xwf". I didn't understand the call for this method. Can
> >> someone
> >> > help me?
> >> >
> >> >
> >>
> airavataAPI.getWorkflowManager().saveWorkflow(getWorkflowComposeContent());
> >> >
> >>  There are two ways to specify what the workflow that needs to be
> >> executed.
> >>    1. Pass the name of the workflow - then the workflow should already
> be
> >>    saved in the registry
> >>    2. Pass the workflow xml as a string
> >> The above command tries to save the workflow to the registry. API user
> has
> >> to pass the workflow xml as a string to the "saveWorkflow" function. The
> >> API will extract the name of the workflow from the xml string & save the
> >> workflow to the registry under that name. This should be a one time
> thing.
> >> That's all it does and has nothing to do with running the workflow at
> this
> >> point.
> >> Then the users can use the 1st way mentioned above to run the workflow
> >> many
> >> times as he/she wants by just passing only the name of the workflow to
> the
> >> runExperiment(...) method.
> >
> >
> > If I understood it correctly, If my workflow is already saved in my
> > register I don't need to run this command again, right?
> >
>
Yep.


>
>
>
> > >
> >> > 2 - The API client transfered my input correctly from my desktop to
> >> > Lonestar, How can I get one output file back from Lonestar to my
> desktop
> >> > using the API? In general, my outputs are just text file and less than
> >> > 500MB.
> >> >
> >> The API client could not have transfered any file from your desktop. I
> >> assume that the Airavata server is running in your machine & the input
> >> paths you may have passed to the experiment are local. The workflow
> >> execution may have taken care of transfering the file in your desktop to
> >> Lonestar. If you require files downloaded back to your desktop then you
> >> need to update your workflow in order to do that.
> >
> >
> >
> Based on your answer "If you require files downloaded back to your desktop
> then you need to update your workflow in order to do that."
>
> What kind of update do I need to do on my workflow?  Do I have to add
> anything in my script that runs on Lonestar?
> My basic understating is that I can setup in my Application Service add
> another Application parameter Output and then provide the type "URI".  What
> is next? Where is the location my output will be saved on my desktop?
>
Well its a little tricky. As far as I know we have grid-ftp capability to
transfer files so as to stage them as input for the next application.
Airavata has no notion of returning files at the end of workflow execution
(to be exact Airavata would not know what files are to transfer anyway
unless the application returns a URI pointing to the file). Theoretically
speaking, if the final result is a file after running a workflow, the user
should manually retrieve the file (using scp/ftp etc.).
Having said that what you could do is to have a new application service
defined in Airavata which points to an application that is in your machine
which accepts a URI as an input (& may or may not have an output). And have
your Lonestar application service output a URI to the file which you want
to be downloaded to your machine & connect that output to the input of your
new application service. (if you are in linux your new application service
could just point to "/bin/echo"). In this way we can trick Airavata to
bring a file back to your machine. Hope I was clear enough... if not please
don't hesitate to ask questions to clarify further.

>
>
>
> > >
> >> > 3 - I noticed in the monitoring part of the code that a Java Thread is
> >> > created and keep the application hanging until the end of the
> monitoring
> >> > execution. Is that possible to create the thread, check the monitor
> >> status
> >> > and kill thread to check the status later in about N minutes? Or Do I
> >> have
> >> > to keep the thread alive during the job execution?
> >> >
> >> Monitoring is used to track the progress of the workflow experiment in
> >> real
> >> time... you can always *not monitor* and check the status at your
> leisure.
> >> Thus the lines,
> >> MonitorWorkflow.monitor(result);
> >> airavataAPI.getExecutionManager().waitForExperimentTermination(result);
> >> are not needed. Important thing to note here is that the command,
> >> airavataAPI.getExecutionManager().runExperiment(workflowName,
> >> workflowInputs)
> >> launches the experiment and returns without waiting for the experiment
> to
> >> finish. Thus the API users should not expect the results to be available
> >> when it tries to read provenance data. Infact it'll take little bit of
> >> time
> >> for the experiment to be registered to the system & initial status of
> the
> >> workflow executions be updated correctly. API users should anticipate
> such
> >> scenarios if they are planning not to use default monitoring or
> >> "waitForExperimentTermination(...)" functionalities.
> >
> >
> >
> My plan is to connect to Airavata-Server to check the status of the job
> every 5 minutes and not keep the socket connection alive until the job
> finishes. Is there any problem doing that?
>
nop.. no problems at all...


> I want to submit more tasks to Lonestar, but now using my Xsede account,
> instead of "ogce".
>
> Thank you for your help Saminda.
>
>
> Please feel free to further ask questions if anything is unclear or if you
> > have any other issues.
> >
> > Regards,
> > Saminda
> >
> >
> > > Thank you,
> > >
> > > Pedro da Silveira
> > >
> >
>

--e0cb4efe2f12dfe5d904d5019205--