airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lahiru Gunathilake <glah...@gmail.com>
Subject Re: Updates to Airavata API version 0.8
Date Fri, 14 Jun 2013 14:54:21 GMT
The one I am working is blocker !

Lahiru


On Fri, Jun 14, 2013 at 10:52 AM, Marlon Pierce <marpierc@iu.edu> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Are the issues blockers for 0.8 release?
>
>
> Marlon
>
>
> On 6/14/13 10:49 AM, Lahiru Gunathilake wrote:
> > Hi Marlon,
> >
> > I have no update about the release, since I've been working with
> > Gary in Ultrascan project. I found some issues to fix for
> > ultrascan, now working on them, so we have to wait until Gary gives
> > the green light to do the release.
> >
> > WDYT ?
> >
> > Regards Lahiru
> >
> >
> > On Fri, Jun 14, 2013 at 10:44 AM, Marlon Pierce <marpierc@iu.edu>
> > wrote:
> >
> > What's the status of the 0.8 release?
> >
> >
> > Marlon
> >
> >
> > On 5/31/13 11:53 AM, Saminda Wijeratne wrote:
> >>>> On Fri, May 31, 2013 at 11:31 AM, Amila Jayasekara
> >>>> <thejaka.amila@gmail.com>wrote:
> >>>>
> >>>>> On Fri, May 31, 2013 at 11:26 AM, Saminda Wijeratne
> >>>>> <samindaw@gmail.com
> >>>>>> wrote:
> >>>>>
> >>>>>> Latest update on AiravataAPI updates for 0.8 version
> >>>>>>
> >>>>>> 1. *Error message from the immediate exception used as
> >>>>>> the error
> >>>>> message
> >>>>>> for the AiravataAPInvocationException*. Earlier it was
> >>>>>> just "Error invoking API". *This change was made so that
> >>>>>> the users of the API can show the error from the
> >>>>>> exception without having to dig in to the
> >>>>> inner
> >>>>>> exception to get a better error message.* 2. *Saving and
> >>>>>> retrieving experiment execution error details*. Earlier
> >>>>>> we were not persisting any execution error details of the
> >>>>>> workflow. Error details are only available only if the
> >>>>>> developers are monitoring experiment at the time of error
> >>>>>> occurs. *This is not feasible if users would only want to
> >>>>>> go through the error details later without doing
> >>>>>> monitoring
> >>>>> from
> >>>>>> the beginning. Therefore we are now persisting the error
> >>>>>> details (not
> >>>>> as
> >>>>>> part of provenance data) of executing workflows (i.e.
> >>>>>> experiments) in
> >>>>> to
> >>>>>> the registry and also retrieving from registry through
> >>>>>> Airavata API (which calls the Registry API). These
> >>>>>> functions are available in the "ExecutionManager" of the
> >>>>>> AiravataAPI and "ProvenanceRegistry" in the RegistryAPI.
> >>>>>> * 3. Persisting & retrieving GFac job data. There are
> >>>>>> many benefits of saving the GRAM data, EC2 data etc. such
> >>>>>> as for debugging, analysis
> >>>>> etc.
> >>>>>> Currently these data are available only through the
> >>>>>> message notifications. *It is not feasible for the same
> >>>>>> reasons mentioned for API change for saving error data.
> >>>>>> Hence the introduction of functions in the API to save &
> >>>>>> retrieve GFac job data.** These functions are available
> >>>>>> as get/setApplicationJob***(...) functions in the
> >>>>>> "ProvenanceManager" of the AiravataAPI and
> >>>>>> "ProvenanceRegistry" in the RegistryAPI.*
> >>>>>>
> >>>>>
> >>>>> May I ask why we need setApplicationJob****() methods in
> >>>>> the API ? As far as I understood job information is
> >>>>> populated by GFac and other internal components. So why do
> >>>>> we need API methods to set those data ? (OR am I missing
> >>>>> something ?)
> >>>>>
> >>>> GFac and Internal components also use the AiravataAPI to
> >>>> populate the job data in the registry.
> >>>>
> >>>>
> >>>>> Thanks Amila
> >>>>>
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> Regards,
> >>>>>>
> >>>>>> Saminda
> >>>>>>
> >>>>>>
> >>>>>> On Tue, May 21, 2013 at 4:59 PM, Raminder Singh <
> >>>>> raminderjsingh@gmail.com
> >>>>>>> wrote:
> >>>>>>
> >>>>>>> Thanks Chathuri, I will test and let you know.
> >>>>>>>
> >>>>>>> Raman On May 21, 2013, at 4:55 PM, Chathuri Wimalasena
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Raman,
> >>>>>>>>
> >>>>>>>> The method Saminda mentioned will work for your case.
> >>>>>>>> I tested with
> >>>>>> your
> >>>>>>>> test class. I fixed deserialization issues from REST
> >>>>>>>> service side.
> >>>>> You
> >>>>>>> will
> >>>>>>>> probably have to update both server and client.
> >>>>>>>>
> >>>>>>>> Regards, Chathuri
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, May 21, 2013 at 3:04 PM, Saminda Wijeratne <
> >>>>> samindaw@gmail.com
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> So far what you are saying is true. If the client
> >>>>>>>>> knows which node
> >>>>>>> failed
> >>>>>>>>> he/she can get the list of errors for that node
> >>>>>>>>> using the getExecutionError(...) function. Use the
> >>>>>>>>> "type" as ALL and leave the gfacJobId as null. I
> >>>>>>>>> couldn;t think of doing this any other better
> >>>>> way
> >>>>>>> than
> >>>>>>>>> introducing a whole lot of functions which could
be
> >>>>>>>>> more
> >>>>>>> annoying/confusing
> >>>>>>>>> to figure-out. I'm welcome for suggestions for
> >>>>>>>>> improvements.
> >>>>>>>>>
> >>>>>>>>> Saminda
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, May 21, 2013 at 2:36 PM, Raminder Singh
<
> >>>>>>> raminderjsingh@gmail.com
> >>>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Thanks Chathuri, the WorkflowExecutionError
> >>>>>>>>>> works. Integrating to
> >>>>>>> Error
> >>>>>>>>>> API brought few questions. We have ErrorTypes
as
> >>>>>>>>>> :
> >>>>>>>>> WorkflowExecutionError,
> >>>>>>>>>> ExperimentExecutionError, NodeExecutionError,
> >>>>> GFacJobExecutionError,
> >>>>>>>>>> ExecutionError . How do we want clients to know
> >>>>>>>>>> which error type to
> >>>>>>>>> query?
> >>>>>>>>>> I ran a job and it failed, now i want to get
> >>>>>>>>>> error. Error occurred
> >>>>> at
> >>>>>>>>> GFac
> >>>>>>>>>> provider level and client does not have any
> >>>>>>>>>> information. Do we
> >>>>> expect
> >>>>>>>>>> client to query all different types?
> >>>>>>>>>>
> >>>>>>>>>> I thing we can have is a method to get all error
> >>>>>>>>>> and return the
> >>>>> Error
> >>>>>>>>> type
> >>>>>>>>>> part of the object to help the client to classify
> >>>>>>>>>> the error
> >>>>> location.
> >>>>>>>>>> Other suggestions?
> >>>>>>>>>>
> >>>>>>>>>> Raminder
> >>>>>>>>>>
> >>>>>>>>>> On May 21, 2013, at 10:57 AM, Chathuri
> >>>>>>>>>> Wimalasena wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi Raman,
> >>>>>>>>>>>
> >>>>>>>>>>> WorkflowExecutionError method not returning
any
> >>>>>>>>>>> data issue should
> >>>>> be
> >>>>>>>>>> fixed
> >>>>>>>>>>> now. Can you get an update and check.
> >>>>>>>>>>>
> >>>>>>>>>>> Regards, Chathuri
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, May 21, 2013 at 10:02 AM, Raminder
> >>>>>>>>>>> Singh <raminderjsingh@gmail.com>wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Thank, these are very useful for API
users.
> >>>>>>>>>>>> Only thing i found
> >>>>>>>>> confusing
> >>>>>>>>>>>> for the user is API managers.
> >>>>>>>>>>>> ProvenanceManager is the one used
> >>>>> to
> >>>>>>>>> get
> >>>>>>>>>>>> experiment status. Users will get FINISHED
> >>>>>>>>>>>> or FAILED status from
> >>>>>> the
> >>>>>>>>>> data
> >>>>>>>>>>>> object. On failure, i was trying to
find a
> >>>>>>>>>>>> method to get the
> >>>>> error
> >>>>>>>>> data
> >>>>>>>>>> but
> >>>>>>>>>>>> i was not able to find as those methods
exist
> >>>>>>>>>>>> in
> >>>>> ExecutionManager.
> >>>>>>>>>>>> According to me, experiment status and
in
> >>>>>>>>>>>> case of failures get
> >>>>>> error
> >>>>>>>>>> need
> >>>>>>>>>>>> to be part of one manager. May be a
> >>>>>>>>>>>> ExperimentManager?  Getting
> >>>>>>> output
> >>>>>>>>>> data
> >>>>>>>>>>>> can be left as part of provenance manages
as
> >>>>>>>>>>>> only few client may
> >>>>>> want
> >>>>>>>>> to
> >>>>>>>>>>>> get the data in few cases. Getting error
> >>>>>>>>>>>> detail in case of error
> >>>>> is
> >>>>>>>>>> obvious
> >>>>>>>>>>>> step.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Other thing can you please explain the
> >>>>>>>>>>>> difference between WorkflowExecutionError,
> >>>>>>>>>>>> ExperimentExecutionError,
> >>>>>> NodeExecutionError,
> >>>>>>>>>>>> GFacExecutionError on a Wiki page for
the
> >>>>>>>>>>>> users. Also add a
> >>>>> method
> >>>>>> to
> >>>>>>>>>> get
> >>>>>>>>>>>> JobID as that is required to get
> >>>>>>>>>>>> GFacExecutionError.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I tested the WorkflowExecutionError
method
> >>>>>>>>>>>> and its not returning
> >>>>>> any
> >>>>>>>>>> data.
> >>>>>>>>>>>> Can you please test that as well?
> >>>>>>>>>>>>
> >>>>>>>>>>>> ExperimentData data =
> >>>>>>>>>>>>
> >>>>> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
> >>>>>>>>>>>>
> >>>>>
> >
> >>>>>
> List<WorkflowExecutionError> experimentExecutionErrors =
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >
> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
> >>>>>>>>>>>>
> >>>>>
> >
> >
> experimentID);
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks Raminder
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On May 20, 2013, at 5:05 PM, Saminda
> >>>>>>>>>>>> Wijeratne wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Following updates will be present
Airavata
> >>>>>>>>>>>>> API from 0.8 release.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 1. *Error message from the immediate
> >>>>>>>>>>>>> exception used as the error
> >>>>>>>>>>>> message
> >>>>>>>>>>>>> for the AiravataAPInvocationException*.
> >>>>>>>>>>>>> Earlier it was just
> >>>>> "Error
> >>>>>>>>>>>>> invoking API". *This change was
made so
> >>>>>>>>>>>>> that the users of the
> >>>>> API
> >>>>>>>>> can
> >>>>>>>>>>>>> show the error from the exception
without
> >>>>>>>>>>>>> having to dig in to
> >>>>> the
> >>>>>>>>>> inner
> >>>>>>>>>>>>> exception to get a better error
message.*
> >>>>>>>>>>>>> 2. *Saving and retrieving experiment
> >>>>>>>>>>>>> execution error details*.
> >>>>>>>>> Earlier
> >>>>>>>>>>>>> we were not persisting any execution
error
> >>>>>>>>>>>>> details of the
> >>>>>> workflow.
> >>>>>>>>>>>> Error
> >>>>>>>>>>>>> details are only available only
if the
> >>>>>>>>>>>>> developers are monitoring
> >>>>>>>>>>>> experiment
> >>>>>>>>>>>>> at the time of error occurs. *This
is not
> >>>>>>>>>>>>> feasible if users
> >>>>> would
> >>>>>>>>> only
> >>>>>>>>>>>>> want to go through the error details
later
> >>>>>>>>>>>>> without doing
> >>>>>> monitoring
> >>>>>>>>>>>> from
> >>>>>>>>>>>>> the beginning. Therefore we are
now
> >>>>>>>>>>>>> persisting the error details
> >>>>>>>>> (not
> >>>>>>>>>>>> as
> >>>>>>>>>>>>> part of provenance data) of executing
> >>>>>>>>>>>>> workflows (i.e.
> >>>>>> experiments)*
> >>>>>>>>> in
> >>>>>>>>>>>>> to the registry and also retrieving
from
> >>>>>>>>>>>>> registry through
> >>>>> Airavata
> >>>>>>>>> API
> >>>>>>>>>>>>> (which calls the Registry API).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> For now we have the functions for
saving
> >>>>>>>>>>>>> and retrieving errors
> >>>>> in
> >>>>>>> the
> >>>>>>>>>>>>> ExecutionManager[1]. We need to
decide if
> >>>>>>>>>>>>> this is the correct
> >>>>>> place
> >>>>>>>>> to
> >>>>>>>>>>>> put
> >>>>>>>>>>>>> these functions. Your thoughts in
this
> >>>>>>>>>>>>> matter are most
> >>>>> welcome...
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks, Saminda
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 1.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >
> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> >
> >
> >
> >
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJRuy4oAAoJEOEgD2XReDo5s34H/1y35R/dwZMDhxZJOjR76wV9
> 441eAf5T/KK6Jh5ipGZVi2BaBqf3Cpudg0RvKdSj/eDwZy+d2pYyjCUFz6XVjC41
> 4expdAUykNcTKwYWEtKFbdDhdAJMeqk1UEGjw9j2BLDvpwBiym38L5WhmNMZi5pM
> RjFShClw3VK/Hdjkx/wTVQVmmMS8vWSUcLEppHzjoYBTLVSapNe2YYR0RwxwQ60i
> nQG+zxDbZqZGobFg+5cYt5xE585FalRHEc66b16Mbor7URzuhfMQuFJK1jWVl3JZ
> dMOyUQMP+JAkGhHfDty9XxeF46lxf6t0ond8y9k9fuHrdEtl1Ub2Sk9xkC17DWw=
> =HTFS
> -----END PGP SIGNATURE-----
>



-- 
System Analyst Programmer
PTI Lab
Indiana University

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message