airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lahiru Gunathilake <glah...@gmail.com>
Subject Re: Updates to Airavata API version 0.8
Date Fri, 14 Jun 2013 15:01:06 GMT
Here is the issue[1].

[1]https://issues.apache.org/jira/browse/AIRAVATA-869

Lahiru


On Fri, Jun 14, 2013 at 10:59 AM, Lahiru Gunathilake <glahiru@gmail.com>wrote:

> I got to know about this yesterday, sorry I couldn't create a jira yet.
> Will do now !
>
> Lahiru
>
>
> On Fri, Jun 14, 2013 at 10:56 AM, Marlon Pierce <marpierc@iu.edu> wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> OK--which Jira task?  Did it get marked as a blocker for 0.8?
>>
>>
>> Marlon
>>
>>
>> On 6/14/13 10:54 AM, Lahiru Gunathilake wrote:
>> > The one I am working is blocker !
>> >
>> > Lahiru
>> >
>> >
>> > On Fri, Jun 14, 2013 at 10:52 AM, Marlon Pierce <marpierc@iu.edu>
>> > wrote:
>> >
>> > Are the issues blockers for 0.8 release?
>> >
>> >
>> > Marlon
>> >
>> >
>> > On 6/14/13 10:49 AM, Lahiru Gunathilake wrote:
>> >>>> Hi Marlon,
>> >>>>
>> >>>> I have no update about the release, since I've been working
>> >>>> with Gary in Ultrascan project. I found some issues to fix
>> >>>> for ultrascan, now working on them, so we have to wait until
>> >>>> Gary gives the green light to do the release.
>> >>>>
>> >>>> WDYT ?
>> >>>>
>> >>>> Regards Lahiru
>> >>>>
>> >>>>
>> >>>> On Fri, Jun 14, 2013 at 10:44 AM, Marlon Pierce
>> >>>> <marpierc@iu.edu> wrote:
>> >>>>
>> >>>> What's the status of the 0.8 release?
>> >>>>
>> >>>>
>> >>>> Marlon
>> >>>>
>> >>>>
>> >>>> On 5/31/13 11:53 AM, Saminda Wijeratne wrote:
>> >>>>>>> On Fri, May 31, 2013 at 11:31 AM, Amila Jayasekara
>> >>>>>>> <thejaka.amila@gmail.com>wrote:
>> >>>>>>>
>> >>>>>>>> On Fri, May 31, 2013 at 11:26 AM, Saminda Wijeratne
>> >>>>>>>> <samindaw@gmail.com
>> >>>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Latest update on AiravataAPI updates for 0.8
>> >>>>>>>>> version
>> >>>>>>>>>
>> >>>>>>>>> 1. *Error message from the immediate exception
used
>> >>>>>>>>> as the error
>> >>>>>>>> message
>> >>>>>>>>> for the AiravataAPInvocationException*. Earlier
it
>> >>>>>>>>> was just "Error invoking API". *This change
was
>> >>>>>>>>> made so that the users of the API can show the
>> >>>>>>>>> error from the exception without having to dig
in
>> >>>>>>>>> to the
>> >>>>>>>> inner
>> >>>>>>>>> exception to get a better error message.* 2.
>> >>>>>>>>> *Saving and retrieving experiment execution
error
>> >>>>>>>>> details*. Earlier we were not persisting any
>> >>>>>>>>> execution error details of the workflow. Error
>> >>>>>>>>> details are only available only if the developers
>> >>>>>>>>> are monitoring experiment at the time of error
>> >>>>>>>>> occurs. *This is not feasible if users would
only
>> >>>>>>>>> want to go through the error details later without
>> >>>>>>>>> doing monitoring
>> >>>>>>>> from
>> >>>>>>>>> the beginning. Therefore we are now persisting
the
>> >>>>>>>>> error details (not
>> >>>>>>>> as
>> >>>>>>>>> part of provenance data) of executing workflows
>> >>>>>>>>> (i.e. experiments) in
>> >>>>>>>> to
>> >>>>>>>>> the registry and also retrieving from registry
>> >>>>>>>>> through Airavata API (which calls the Registry
>> >>>>>>>>> API). These functions are available in the
>> >>>>>>>>> "ExecutionManager" of the AiravataAPI and
>> >>>>>>>>> "ProvenanceRegistry" in the RegistryAPI. * 3.
>> >>>>>>>>> Persisting & retrieving GFac job data. There
are
>> >>>>>>>>> many benefits of saving the GRAM data, EC2 data
>> >>>>>>>>> etc. such as for debugging, analysis
>> >>>>>>>> etc.
>> >>>>>>>>> Currently these data are available only through
>> >>>>>>>>> the message notifications. *It is not feasible
for
>> >>>>>>>>> the same reasons mentioned for API change for
>> >>>>>>>>> saving error data. Hence the introduction of
>> >>>>>>>>> functions in the API to save & retrieve
GFac job
>> >>>>>>>>> data.** These functions are available as
>> >>>>>>>>> get/setApplicationJob***(...) functions in the
>> >>>>>>>>> "ProvenanceManager" of the AiravataAPI and
>> >>>>>>>>> "ProvenanceRegistry" in the RegistryAPI.*
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>> May I ask why we need setApplicationJob****() methods
>> >>>>>>>> in the API ? As far as I understood job information
>> >>>>>>>> is populated by GFac and other internal components.
>> >>>>>>>> So why do we need API methods to set those data
? (OR
>> >>>>>>>> am I missing something ?)
>> >>>>>>>>
>> >>>>>>> GFac and Internal components also use the AiravataAPI
>> >>>>>>> to populate the job data in the registry.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> Thanks Amila
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> Regards,
>> >>>>>>>>>
>> >>>>>>>>> Saminda
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Tue, May 21, 2013 at 4:59 PM, Raminder Singh
<
>> >>>>>>>> raminderjsingh@gmail.com
>> >>>>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> Thanks Chathuri, I will test and let you
know.
>> >>>>>>>>>>
>> >>>>>>>>>> Raman On May 21, 2013, at 4:55 PM, Chathuri
>> >>>>>>>>>> Wimalasena wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Hi Raman,
>> >>>>>>>>>>>
>> >>>>>>>>>>> The method Saminda mentioned will work
for your
>> >>>>>>>>>>> case. I tested with
>> >>>>>>>>> your
>> >>>>>>>>>>> test class. I fixed deserialization
issues from
>> >>>>>>>>>>> REST service side.
>> >>>>>>>> You
>> >>>>>>>>>> will
>> >>>>>>>>>>> probably have to update both server
and
>> >>>>>>>>>>> client.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Regards, Chathuri
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Tue, May 21, 2013 at 3:04 PM, Saminda
>> >>>>>>>>>>> Wijeratne <
>> >>>>>>>> samindaw@gmail.com
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> So far what you are saying is true.
If the
>> >>>>>>>>>>>> client knows which node
>> >>>>>>>>>> failed
>> >>>>>>>>>>>> he/she can get the list of errors
for that
>> >>>>>>>>>>>> node using the getExecutionError(...)
>> >>>>>>>>>>>> function. Use the "type" as ALL
and leave the
>> >>>>>>>>>>>> gfacJobId as null. I couldn;t think
of doing
>> >>>>>>>>>>>> this any other better
>> >>>>>>>> way
>> >>>>>>>>>> than
>> >>>>>>>>>>>> introducing a whole lot of functions
which
>> >>>>>>>>>>>> could be more
>> >>>>>>>>>> annoying/confusing
>> >>>>>>>>>>>> to figure-out. I'm welcome for suggestions
>> >>>>>>>>>>>> for improvements.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Saminda
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Tue, May 21, 2013 at 2:36 PM,
Raminder
>> >>>>>>>>>>>> Singh <
>> >>>>>>>>>> raminderjsingh@gmail.com
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Thanks Chathuri, the
>> >>>>>>>>>>>>> WorkflowExecutionError works.
Integrating
>> >>>>>>>>>>>>> to
>> >>>>>>>>>> Error
>> >>>>>>>>>>>>> API brought few questions. We
have
>> >>>>>>>>>>>>> ErrorTypes as :
>> >>>>>>>>>>>> WorkflowExecutionError,
>> >>>>>>>>>>>>> ExperimentExecutionError,
>> >>>>>>>>>>>>> NodeExecutionError,
>> >>>>>>>> GFacJobExecutionError,
>> >>>>>>>>>>>>> ExecutionError . How do we want
clients to
>> >>>>>>>>>>>>> know which error type to
>> >>>>>>>>>>>> query?
>> >>>>>>>>>>>>> I ran a job and it failed, now
i want to
>> >>>>>>>>>>>>> get error. Error occurred
>> >>>>>>>> at
>> >>>>>>>>>>>> GFac
>> >>>>>>>>>>>>> provider level and client does
not have
>> >>>>>>>>>>>>> any information. Do we
>> >>>>>>>> expect
>> >>>>>>>>>>>>> client to query all different
types?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I thing we can have is a method
to get all
>> >>>>>>>>>>>>> error and return the
>> >>>>>>>> Error
>> >>>>>>>>>>>> type
>> >>>>>>>>>>>>> part of the object to help the
client to
>> >>>>>>>>>>>>> classify the error
>> >>>>>>>> location.
>> >>>>>>>>>>>>> Other suggestions?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Raminder
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On May 21, 2013, at 10:57 AM,
Chathuri
>> >>>>>>>>>>>>> Wimalasena wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Hi Raman,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> WorkflowExecutionError method
not
>> >>>>>>>>>>>>>> returning any data issue
should
>> >>>>>>>> be
>> >>>>>>>>>>>>> fixed
>> >>>>>>>>>>>>>> now. Can you get an update
and check.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Regards, Chathuri
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Tue, May 21, 2013 at
10:02 AM,
>> >>>>>>>>>>>>>> Raminder Singh
>> >>>>>>>>>>>>>> <raminderjsingh@gmail.com>wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Thank, these are very
useful for API
>> >>>>>>>>>>>>>>> users. Only thing i
found
>> >>>>>>>>>>>> confusing
>> >>>>>>>>>>>>>>> for the user is API
managers.
>> >>>>>>>>>>>>>>> ProvenanceManager is
the one used
>> >>>>>>>> to
>> >>>>>>>>>>>> get
>> >>>>>>>>>>>>>>> experiment status. Users
will get
>> >>>>>>>>>>>>>>> FINISHED or FAILED status
from
>> >>>>>>>>> the
>> >>>>>>>>>>>>> data
>> >>>>>>>>>>>>>>> object. On failure,
i was trying to
>> >>>>>>>>>>>>>>> find a method to get
the
>> >>>>>>>> error
>> >>>>>>>>>>>> data
>> >>>>>>>>>>>>> but
>> >>>>>>>>>>>>>>> i was not able to find
as those methods
>> >>>>>>>>>>>>>>> exist in
>> >>>>>>>> ExecutionManager.
>> >>>>>>>>>>>>>>> According to me, experiment
status and
>> >>>>>>>>>>>>>>> in case of failures
get
>> >>>>>>>>> error
>> >>>>>>>>>>>>> need
>> >>>>>>>>>>>>>>> to be part of one manager.
May be a
>> >>>>>>>>>>>>>>> ExperimentManager? 
Getting
>> >>>>>>>>>> output
>> >>>>>>>>>>>>> data
>> >>>>>>>>>>>>>>> can be left as part
of provenance
>> >>>>>>>>>>>>>>> manages as only few
client may
>> >>>>>>>>> want
>> >>>>>>>>>>>> to
>> >>>>>>>>>>>>>>> get the data in few
cases. Getting
>> >>>>>>>>>>>>>>> error detail in case
of error
>> >>>>>>>> is
>> >>>>>>>>>>>>> obvious
>> >>>>>>>>>>>>>>> step.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Other thing can you
please explain the
>> >>>>>>>>>>>>>>> difference between
>> >>>>>>>>>>>>>>> WorkflowExecutionError,
>> >>>>>>>>>>>>>>> ExperimentExecutionError,
>> >>>>>>>>> NodeExecutionError,
>> >>>>>>>>>>>>>>> GFacExecutionError on
a Wiki page for
>> >>>>>>>>>>>>>>> the users. Also add
a
>> >>>>>>>> method
>> >>>>>>>>> to
>> >>>>>>>>>>>>> get
>> >>>>>>>>>>>>>>> JobID as that is required
to get
>> >>>>>>>>>>>>>>> GFacExecutionError.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> I tested the WorkflowExecutionError
>> >>>>>>>>>>>>>>> method and its not returning
>> >>>>>>>>> any
>> >>>>>>>>>>>>> data.
>> >>>>>>>>>>>>>>> Can you please test
that as well?
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> ExperimentData data
=
>> >>>>>>>>>>>>>>>
>> >>>>>>>>
>> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
>> >>>>>>>>>>>>>>>
>> >>>>>>>>
>> >>>>
>> >>>>>>>>
>> >
>> >>>>>>>>
>> List<WorkflowExecutionError> experimentExecutionErrors =
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>
>> >
>> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
>> >>>>>>>>>>>>>>>
>> >>>>>>>>
>> >>>>
>> >>>>
>> >
>> >
>> experimentID);
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Thanks Raminder
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On May 20, 2013, at
5:05 PM, Saminda
>> >>>>>>>>>>>>>>> Wijeratne wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Following updates
will be present
>> >>>>>>>>>>>>>>>> Airavata API from
0.8 release.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> 1. *Error message
from the immediate
>> >>>>>>>>>>>>>>>> exception used as
the error
>> >>>>>>>>>>>>>>> message
>> >>>>>>>>>>>>>>>> for the
>> >>>>>>>>>>>>>>>> AiravataAPInvocationException*.
>> >>>>>>>>>>>>>>>> Earlier it was just
>> >>>>>>>> "Error
>> >>>>>>>>>>>>>>>> invoking API". *This
change was made
>> >>>>>>>>>>>>>>>> so that the users
of the
>> >>>>>>>> API
>> >>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>> show the error from
the exception
>> >>>>>>>>>>>>>>>> without having to
dig in to
>> >>>>>>>> the
>> >>>>>>>>>>>>> inner
>> >>>>>>>>>>>>>>>> exception to get
a better error
>> >>>>>>>>>>>>>>>> message.* 2. *Saving
and retrieving
>> >>>>>>>>>>>>>>>> experiment execution
error details*.
>> >>>>>>>>>>>> Earlier
>> >>>>>>>>>>>>>>>> we were not persisting
any execution
>> >>>>>>>>>>>>>>>> error details of
the
>> >>>>>>>>> workflow.
>> >>>>>>>>>>>>>>> Error
>> >>>>>>>>>>>>>>>> details are only
available only if
>> >>>>>>>>>>>>>>>> the developers are
monitoring
>> >>>>>>>>>>>>>>> experiment
>> >>>>>>>>>>>>>>>> at the time of error
occurs. *This is
>> >>>>>>>>>>>>>>>> not feasible if
users
>> >>>>>>>> would
>> >>>>>>>>>>>> only
>> >>>>>>>>>>>>>>>> want to go through
the error details
>> >>>>>>>>>>>>>>>> later without doing
>> >>>>>>>>> monitoring
>> >>>>>>>>>>>>>>> from
>> >>>>>>>>>>>>>>>> the beginning. Therefore
we are now
>> >>>>>>>>>>>>>>>> persisting the error
details
>> >>>>>>>>>>>> (not
>> >>>>>>>>>>>>>>> as
>> >>>>>>>>>>>>>>>> part of provenance
data) of
>> >>>>>>>>>>>>>>>> executing workflows
(i.e.
>> >>>>>>>>> experiments)*
>> >>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>> to the registry
and also retrieving
>> >>>>>>>>>>>>>>>> from registry through
>> >>>>>>>> Airavata
>> >>>>>>>>>>>> API
>> >>>>>>>>>>>>>>>> (which calls the
Registry API).
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> For now we have
the functions for
>> >>>>>>>>>>>>>>>> saving and retrieving
errors
>> >>>>>>>> in
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>>> ExecutionManager[1].
We need to
>> >>>>>>>>>>>>>>>> decide if this is
the correct
>> >>>>>>>>> place
>> >>>>>>>>>>>> to
>> >>>>>>>>>>>>>>> put
>> >>>>>>>>>>>>>>>> these functions.
Your thoughts in
>> >>>>>>>>>>>>>>>> this matter are
most
>> >>>>>>>> welcome...
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Thanks, Saminda
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> 1.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>
>> >
>> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>
>> >
>> >
>> >
>> >
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
>> Comment: GPGTools - http://gpgtools.org
>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>>
>> iQEcBAEBAgAGBQJRuy8SAAoJEOEgD2XReDo53w0H/2wxRPUk2TjXqwRnCdVuSYQL
>> uAA6I/TCp8j1+uugNNcCKApi3npo8FTtZJDIWGDACzpzZg/zsTOp583DldzpXZXL
>> e6N2Bzb+dz5woWYG/i9/EBq585ErH/e9ieDgRP0MfrryjIoNj+AKcCtutIWmfjHt
>> 1E3MhpXO0yfYxiUdiYtH1nK/O5qJHhVc7foZj+Q5XiCza9Z8p3BPcKY9l5AUWMUE
>> N/4HZ5mat78bJriz9qYejscVDDadfagkYAYoTQ3srKlJdXBloeL4Yhw7kE4P5gU6
>> MVFqlRIC18Fj5Ugv0bj1rMUMw7vPWiQe7fTGG2hYyCGTfsu81r9V1/3FQtyRkY0=
>> =roFX
>> -----END PGP SIGNATURE-----
>>
>
>
>
> --
> System Analyst Programmer
> PTI Lab
> Indiana University
>



-- 
System Analyst Programmer
PTI Lab
Indiana University

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message