airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saminda Wijeratne <samin...@gmail.com>
Subject Re: Updates to Airavata API version 0.8
Date Fri, 31 May 2013 15:53:20 GMT
On Fri, May 31, 2013 at 11:31 AM, Amila Jayasekara
<thejaka.amila@gmail.com>wrote:

> On Fri, May 31, 2013 at 11:26 AM, Saminda Wijeratne <samindaw@gmail.com
> >wrote:
>
> > Latest update on AiravataAPI updates for 0.8 version
> >
> >    1. *Error message from the immediate exception used as the error
> message
> >    for the AiravataAPInvocationException*. Earlier it was just "Error
> >    invoking API". *This change was made so that the users of the API can
> >    show the error from the exception without having to dig in to the
> inner
> >    exception to get a better error message.*
> >    2. *Saving and retrieving experiment execution error details*. Earlier
> >    we were not persisting any execution error details of the workflow.
> > Error
> >    details are only available only if the developers are monitoring
> > experiment
> >    at the time of error occurs. *This is not feasible if users would only
> >    want to go through the error details later without doing monitoring
> from
> >    the beginning. Therefore we are now persisting the error details (not
> as
> >    part of provenance data) of executing workflows (i.e. experiments) in
> to
> >    the registry and also retrieving from registry through Airavata API
> > (which
> >    calls the Registry API). These functions are available in the
> >    "ExecutionManager" of the AiravataAPI and "ProvenanceRegistry" in the
> >    RegistryAPI.
> >    *
> >    3. Persisting & retrieving GFac job data. There are many benefits of
> >    saving the GRAM data, EC2 data etc. such as for debugging, analysis
> etc.
> >    Currently these data are available only through the message
> > notifications.
> >    *It is not feasible for the same reasons mentioned for API change for
> >    saving error data. Hence the introduction of functions in the API to
> > save &
> >    retrieve GFac job data.** These functions are available as
> >    get/setApplicationJob***(...) functions in the "ProvenanceManager" of
> > the
> >    AiravataAPI and "ProvenanceRegistry" in the RegistryAPI.*
> >
>
> May I ask why we need setApplicationJob****() methods in the API ?
> As far as I understood job information is populated by GFac and other
> internal components. So why do we need API methods to set those data ? (OR
> am I missing something ?)
>
GFac and Internal components also use the AiravataAPI to populate the job
data in the registry.


> Thanks
> Amila
>
>
> >
> >
> > Regards,
> >
> > Saminda
> >
> >
> > On Tue, May 21, 2013 at 4:59 PM, Raminder Singh <
> raminderjsingh@gmail.com
> > >wrote:
> >
> > > Thanks Chathuri, I will test and let you know.
> > >
> > > Raman
> > > On May 21, 2013, at 4:55 PM, Chathuri Wimalasena wrote:
> > >
> > > > Hi Raman,
> > > >
> > > > The method Saminda mentioned will work for your case. I tested with
> > your
> > > > test class. I fixed deserialization issues from REST service side.
> You
> > > will
> > > > probably have to update both server and client.
> > > >
> > > > Regards,
> > > > Chathuri
> > > >
> > > >
> > > > On Tue, May 21, 2013 at 3:04 PM, Saminda Wijeratne <
> samindaw@gmail.com
> > > >wrote:
> > > >
> > > >> So far what you are saying is true. If the client knows which node
> > > failed
> > > >> he/she can get the list of errors for that node using the
> > > >> getExecutionError(...) function. Use the "type" as ALL and leave the
> > > >> gfacJobId as null. I couldn;t think of doing this any other better
> way
> > > than
> > > >> introducing a whole lot of functions which could be more
> > > annoying/confusing
> > > >> to figure-out. I'm welcome for suggestions for improvements.
> > > >>
> > > >> Saminda
> > > >>
> > > >>
> > > >> On Tue, May 21, 2013 at 2:36 PM, Raminder Singh <
> > > raminderjsingh@gmail.com
> > > >>> wrote:
> > > >>
> > > >>> Thanks Chathuri, the WorkflowExecutionError works.  Integrating
to
> > > Error
> > > >>> API brought few questions. We have ErrorTypes as :
> > > >> WorkflowExecutionError,
> > > >>> ExperimentExecutionError, NodeExecutionError,
> GFacJobExecutionError,
> > > >>> ExecutionError . How do we want clients to know which error type
to
> > > >> query?
> > > >>> I ran a job and it failed, now i want to get error. Error occurred
> at
> > > >> GFac
> > > >>> provider level and client does not have any information. Do we
> expect
> > > >>> client to query all different types?
> > > >>>
> > > >>> I thing we can have is a method to get all error and return the
> Error
> > > >> type
> > > >>> part of the object to help the client to classify the error
> location.
> > > >>> Other suggestions?
> > > >>>
> > > >>> Raminder
> > > >>>
> > > >>> On May 21, 2013, at 10:57 AM, Chathuri Wimalasena wrote:
> > > >>>
> > > >>>> Hi Raman,
> > > >>>>
> > > >>>> WorkflowExecutionError method not returning any data issue
should
> be
> > > >>> fixed
> > > >>>> now. Can you get an update and check.
> > > >>>>
> > > >>>> Regards,
> > > >>>> Chathuri
> > > >>>>
> > > >>>>
> > > >>>> On Tue, May 21, 2013 at 10:02 AM, Raminder Singh
> > > >>>> <raminderjsingh@gmail.com>wrote:
> > > >>>>
> > > >>>>> Thank, these are very useful for API users. Only thing
i found
> > > >> confusing
> > > >>>>> for the user is API managers.  ProvenanceManager is the
one used
> to
> > > >> get
> > > >>>>> experiment status. Users will get FINISHED or FAILED status
from
> > the
> > > >>> data
> > > >>>>> object. On failure, i was trying to find a method to get
the
> error
> > > >> data
> > > >>> but
> > > >>>>> i was not able to find as those methods exist in
> ExecutionManager.
> > > >>>>> According to me, experiment status and in case of failures
get
> > error
> > > >>> need
> > > >>>>> to be part of one manager. May be a ExperimentManager?
 Getting
> > > output
> > > >>> data
> > > >>>>> can be left as part of provenance manages as only few
client may
> > want
> > > >> to
> > > >>>>> get the data in few cases. Getting error detail in case
of error
> is
> > > >>> obvious
> > > >>>>> step.
> > > >>>>>
> > > >>>>> Other thing can you please explain the difference between
> > > >>>>> WorkflowExecutionError, ExperimentExecutionError,
> > NodeExecutionError,
> > > >>>>> GFacExecutionError on a Wiki page for the users. Also
add a
> method
> > to
> > > >>> get
> > > >>>>> JobID as that is required to get GFacExecutionError.
> > > >>>>>
> > > >>>>> I tested the WorkflowExecutionError method and its not
returning
> > any
> > > >>> data.
> > > >>>>> Can you please test that as well?
> > > >>>>>
> > > >>>>> ExperimentData data =
> > > >>>>>
> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
> > > >>>>> List<WorkflowExecutionError> experimentExecutionErrors
=
> > > >>>>>
> > > >>>
> > > >>
> > >
> >
> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
> > > >>>>> experimentID);
> > > >>>>>
> > > >>>>> Thanks
> > > >>>>> Raminder
> > > >>>>>
> > > >>>>>
> > > >>>>> On May 20, 2013, at 5:05 PM, Saminda Wijeratne wrote:
> > > >>>>>
> > > >>>>>> Following updates will be present Airavata API from
0.8 release.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> 1. *Error message from the immediate exception used
as the error
> > > >>>>> message
> > > >>>>>> for the AiravataAPInvocationException*. Earlier it
was just
> "Error
> > > >>>>>> invoking API". *This change was made so that the users
of the
> API
> > > >> can
> > > >>>>>> show the error from the exception without having to
dig in to
> the
> > > >>> inner
> > > >>>>>> exception to get a better error message.*
> > > >>>>>> 2. *Saving and retrieving experiment execution error
details*.
> > > >> Earlier
> > > >>>>>> we were not persisting any execution error details
of the
> > workflow.
> > > >>>>> Error
> > > >>>>>> details are only available only if the developers
are monitoring
> > > >>>>> experiment
> > > >>>>>> at the time of error occurs. *This is not feasible
if users
> would
> > > >> only
> > > >>>>>> want to go through the error details later without
doing
> > monitoring
> > > >>>>> from
> > > >>>>>> the beginning. Therefore we are now persisting the
error details
> > > >> (not
> > > >>>>> as
> > > >>>>>> part of provenance data) of executing workflows (i.e.
> > experiments)*
> > > >> in
> > > >>>>>> to the registry and also retrieving from registry
through
> Airavata
> > > >> API
> > > >>>>>> (which calls the Registry API).
> > > >>>>>>
> > > >>>>>> For now we have the functions for saving and retrieving
errors
> in
> > > the
> > > >>>>>> ExecutionManager[1]. We need to decide if this is
the correct
> > place
> > > >> to
> > > >>>>> put
> > > >>>>>> these functions. Your thoughts in this matter are
most
> welcome...
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Thanks,
> > > >>>>>> Saminda
> > > >>>>>>
> > > >>>>>> 1.
> > > >>>>>>
> > > >>>>>
> > > >>>
> > > >>
> > >
> >
> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
> > > >>>>>
> > > >>>>>
> > > >>>
> > > >>>
> > > >>
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message