airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lahiru Gunathilake <glah...@gmail.com>
Subject Re: Updates to Airavata API version 0.8
Date Fri, 14 Jun 2013 14:49:34 GMT
Hi Marlon,

I have no update about the release, since I've been working with Gary in
Ultrascan project. I found some issues to fix for ultrascan, now working on
them, so we have to wait until Gary gives the green light to do the release.

WDYT ?

Regards
Lahiru


On Fri, Jun 14, 2013 at 10:44 AM, Marlon Pierce <marpierc@iu.edu> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> What's the status of the 0.8 release?
>
>
> Marlon
>
>
> On 5/31/13 11:53 AM, Saminda Wijeratne wrote:
> > On Fri, May 31, 2013 at 11:31 AM, Amila Jayasekara
> > <thejaka.amila@gmail.com>wrote:
> >
> >> On Fri, May 31, 2013 at 11:26 AM, Saminda Wijeratne
> >> <samindaw@gmail.com
> >>> wrote:
> >>
> >>> Latest update on AiravataAPI updates for 0.8 version
> >>>
> >>> 1. *Error message from the immediate exception used as the
> >>> error
> >> message
> >>> for the AiravataAPInvocationException*. Earlier it was just
> >>> "Error invoking API". *This change was made so that the users
> >>> of the API can show the error from the exception without having
> >>> to dig in to the
> >> inner
> >>> exception to get a better error message.* 2. *Saving and
> >>> retrieving experiment execution error details*. Earlier we were
> >>> not persisting any execution error details of the workflow.
> >>> Error details are only available only if the developers are
> >>> monitoring experiment at the time of error occurs. *This is not
> >>> feasible if users would only want to go through the error
> >>> details later without doing monitoring
> >> from
> >>> the beginning. Therefore we are now persisting the error
> >>> details (not
> >> as
> >>> part of provenance data) of executing workflows (i.e.
> >>> experiments) in
> >> to
> >>> the registry and also retrieving from registry through Airavata
> >>> API (which calls the Registry API). These functions are
> >>> available in the "ExecutionManager" of the AiravataAPI and
> >>> "ProvenanceRegistry" in the RegistryAPI. * 3. Persisting &
> >>> retrieving GFac job data. There are many benefits of saving the
> >>> GRAM data, EC2 data etc. such as for debugging, analysis
> >> etc.
> >>> Currently these data are available only through the message
> >>> notifications. *It is not feasible for the same reasons
> >>> mentioned for API change for saving error data. Hence the
> >>> introduction of functions in the API to save & retrieve GFac
> >>> job data.** These functions are available as
> >>> get/setApplicationJob***(...) functions in the
> >>> "ProvenanceManager" of the AiravataAPI and "ProvenanceRegistry"
> >>> in the RegistryAPI.*
> >>>
> >>
> >> May I ask why we need setApplicationJob****() methods in the API
> >> ? As far as I understood job information is populated by GFac and
> >> other internal components. So why do we need API methods to set
> >> those data ? (OR am I missing something ?)
> >>
> > GFac and Internal components also use the AiravataAPI to populate
> > the job data in the registry.
> >
> >
> >> Thanks Amila
> >>
> >>
> >>>
> >>>
> >>> Regards,
> >>>
> >>> Saminda
> >>>
> >>>
> >>> On Tue, May 21, 2013 at 4:59 PM, Raminder Singh <
> >> raminderjsingh@gmail.com
> >>>> wrote:
> >>>
> >>>> Thanks Chathuri, I will test and let you know.
> >>>>
> >>>> Raman On May 21, 2013, at 4:55 PM, Chathuri Wimalasena
> >>>> wrote:
> >>>>
> >>>>> Hi Raman,
> >>>>>
> >>>>> The method Saminda mentioned will work for your case. I
> >>>>> tested with
> >>> your
> >>>>> test class. I fixed deserialization issues from REST
> >>>>> service side.
> >> You
> >>>> will
> >>>>> probably have to update both server and client.
> >>>>>
> >>>>> Regards, Chathuri
> >>>>>
> >>>>>
> >>>>> On Tue, May 21, 2013 at 3:04 PM, Saminda Wijeratne <
> >> samindaw@gmail.com
> >>>>> wrote:
> >>>>>
> >>>>>> So far what you are saying is true. If the client knows
> >>>>>> which node
> >>>> failed
> >>>>>> he/she can get the list of errors for that node using
> >>>>>> the getExecutionError(...) function. Use the "type" as
> >>>>>> ALL and leave the gfacJobId as null. I couldn;t think of
> >>>>>> doing this any other better
> >> way
> >>>> than
> >>>>>> introducing a whole lot of functions which could be more
> >>>> annoying/confusing
> >>>>>> to figure-out. I'm welcome for suggestions for
> >>>>>> improvements.
> >>>>>>
> >>>>>> Saminda
> >>>>>>
> >>>>>>
> >>>>>> On Tue, May 21, 2013 at 2:36 PM, Raminder Singh <
> >>>> raminderjsingh@gmail.com
> >>>>>>> wrote:
> >>>>>>
> >>>>>>> Thanks Chathuri, the WorkflowExecutionError works.
> >>>>>>> Integrating to
> >>>> Error
> >>>>>>> API brought few questions. We have ErrorTypes as :
> >>>>>> WorkflowExecutionError,
> >>>>>>> ExperimentExecutionError, NodeExecutionError,
> >> GFacJobExecutionError,
> >>>>>>> ExecutionError . How do we want clients to know which
> >>>>>>> error type to
> >>>>>> query?
> >>>>>>> I ran a job and it failed, now i want to get error.
> >>>>>>> Error occurred
> >> at
> >>>>>> GFac
> >>>>>>> provider level and client does not have any
> >>>>>>> information. Do we
> >> expect
> >>>>>>> client to query all different types?
> >>>>>>>
> >>>>>>> I thing we can have is a method to get all error and
> >>>>>>> return the
> >> Error
> >>>>>> type
> >>>>>>> part of the object to help the client to classify the
> >>>>>>> error
> >> location.
> >>>>>>> Other suggestions?
> >>>>>>>
> >>>>>>> Raminder
> >>>>>>>
> >>>>>>> On May 21, 2013, at 10:57 AM, Chathuri Wimalasena
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Raman,
> >>>>>>>>
> >>>>>>>> WorkflowExecutionError method not returning any data
> >>>>>>>> issue should
> >> be
> >>>>>>> fixed
> >>>>>>>> now. Can you get an update and check.
> >>>>>>>>
> >>>>>>>> Regards, Chathuri
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, May 21, 2013 at 10:02 AM, Raminder Singh
> >>>>>>>> <raminderjsingh@gmail.com>wrote:
> >>>>>>>>
> >>>>>>>>> Thank, these are very useful for API users. Only
> >>>>>>>>> thing i found
> >>>>>> confusing
> >>>>>>>>> for the user is API managers.  ProvenanceManager
is
> >>>>>>>>> the one used
> >> to
> >>>>>> get
> >>>>>>>>> experiment status. Users will get FINISHED or
> >>>>>>>>> FAILED status from
> >>> the
> >>>>>>> data
> >>>>>>>>> object. On failure, i was trying to find a method
> >>>>>>>>> to get the
> >> error
> >>>>>> data
> >>>>>>> but
> >>>>>>>>> i was not able to find as those methods exist in
> >> ExecutionManager.
> >>>>>>>>> According to me, experiment status and in case of
> >>>>>>>>> failures get
> >>> error
> >>>>>>> need
> >>>>>>>>> to be part of one manager. May be a
> >>>>>>>>> ExperimentManager?  Getting
> >>>> output
> >>>>>>> data
> >>>>>>>>> can be left as part of provenance manages as only
> >>>>>>>>> few client may
> >>> want
> >>>>>> to
> >>>>>>>>> get the data in few cases. Getting error detail
in
> >>>>>>>>> case of error
> >> is
> >>>>>>> obvious
> >>>>>>>>> step.
> >>>>>>>>>
> >>>>>>>>> Other thing can you please explain the difference
> >>>>>>>>> between WorkflowExecutionError,
> >>>>>>>>> ExperimentExecutionError,
> >>> NodeExecutionError,
> >>>>>>>>> GFacExecutionError on a Wiki page for the users.
> >>>>>>>>> Also add a
> >> method
> >>> to
> >>>>>>> get
> >>>>>>>>> JobID as that is required to get
> >>>>>>>>> GFacExecutionError.
> >>>>>>>>>
> >>>>>>>>> I tested the WorkflowExecutionError method and its
> >>>>>>>>> not returning
> >>> any
> >>>>>>> data.
> >>>>>>>>> Can you please test that as well?
> >>>>>>>>>
> >>>>>>>>> ExperimentData data =
> >>>>>>>>>
> >> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
> >>>>>>>>>
> >>
> List<WorkflowExecutionError> experimentExecutionErrors =
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
> >>>>>>>>>
> >>
> experimentID);
> >>>>>>>>>
> >>>>>>>>> Thanks Raminder
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On May 20, 2013, at 5:05 PM, Saminda Wijeratne
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Following updates will be present Airavata API
> >>>>>>>>>> from 0.8 release.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> 1. *Error message from the immediate exception
> >>>>>>>>>> used as the error
> >>>>>>>>> message
> >>>>>>>>>> for the AiravataAPInvocationException*. Earlier
> >>>>>>>>>> it was just
> >> "Error
> >>>>>>>>>> invoking API". *This change was made so that
the
> >>>>>>>>>> users of the
> >> API
> >>>>>> can
> >>>>>>>>>> show the error from the exception without having
> >>>>>>>>>> to dig in to
> >> the
> >>>>>>> inner
> >>>>>>>>>> exception to get a better error message.* 2.
> >>>>>>>>>> *Saving and retrieving experiment execution
error
> >>>>>>>>>> details*.
> >>>>>> Earlier
> >>>>>>>>>> we were not persisting any execution error
> >>>>>>>>>> details of the
> >>> workflow.
> >>>>>>>>> Error
> >>>>>>>>>> details are only available only if the developers
> >>>>>>>>>> are monitoring
> >>>>>>>>> experiment
> >>>>>>>>>> at the time of error occurs. *This is not
> >>>>>>>>>> feasible if users
> >> would
> >>>>>> only
> >>>>>>>>>> want to go through the error details later
> >>>>>>>>>> without doing
> >>> monitoring
> >>>>>>>>> from
> >>>>>>>>>> the beginning. Therefore we are now persisting
> >>>>>>>>>> the error details
> >>>>>> (not
> >>>>>>>>> as
> >>>>>>>>>> part of provenance data) of executing workflows
> >>>>>>>>>> (i.e.
> >>> experiments)*
> >>>>>> in
> >>>>>>>>>> to the registry and also retrieving from registry
> >>>>>>>>>> through
> >> Airavata
> >>>>>> API
> >>>>>>>>>> (which calls the Registry API).
> >>>>>>>>>>
> >>>>>>>>>> For now we have the functions for saving and
> >>>>>>>>>> retrieving errors
> >> in
> >>>> the
> >>>>>>>>>> ExecutionManager[1]. We need to decide if this
is
> >>>>>>>>>> the correct
> >>> place
> >>>>>> to
> >>>>>>>>> put
> >>>>>>>>>> these functions. Your thoughts in this matter
are
> >>>>>>>>>> most
> >> welcome...
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Thanks, Saminda
> >>>>>>>>>>
> >>>>>>>>>> 1.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>
> >>
> >
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJRuyxqAAoJEOEgD2XReDo54nwIAKjDjVqPk6YldnJD+T91k2HU
> Yj/VcvH/XAxvoZOu9zGpcUFNuLPbSiJEJes9Tx3Wk63k8v+l4rxTTRB5rWihbyYn
> xPEKtSnD8sCAiuFXlRHWLygzE80OVhC9Q43xkrnjjubKaWKp3ZgU1uF1ZL6LiTZb
> R83+Yl2WCdAAckX1W+567APk202nw93GLKGjmRcuj2mBPKgWXzzD7AX3cQX6pt9e
> g93PKGWJWdk/kb6YlhjkEfKH2RxJS/AgZvUYSAmf1ELL+siGqxEEOg7bwWwpfLNd
> NsjUr/rN88EcGfUKxYb4fSh1zGceqp02rWG2TwufZEIVaF8WjztY6CsE2TgfS0M=
> =vpMl
> -----END PGP SIGNATURE-----
>



-- 
System Analyst Programmer
PTI Lab
Indiana University

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message