airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marlon Pierce <marpi...@iu.edu>
Subject Re: Updates to Airavata API version 0.8
Date Fri, 14 Jun 2013 14:44:58 GMT
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

What's the status of the 0.8 release?


Marlon


On 5/31/13 11:53 AM, Saminda Wijeratne wrote:
> On Fri, May 31, 2013 at 11:31 AM, Amila Jayasekara 
> <thejaka.amila@gmail.com>wrote:
> 
>> On Fri, May 31, 2013 at 11:26 AM, Saminda Wijeratne
>> <samindaw@gmail.com
>>> wrote:
>> 
>>> Latest update on AiravataAPI updates for 0.8 version
>>> 
>>> 1. *Error message from the immediate exception used as the
>>> error
>> message
>>> for the AiravataAPInvocationException*. Earlier it was just
>>> "Error invoking API". *This change was made so that the users
>>> of the API can show the error from the exception without having
>>> to dig in to the
>> inner
>>> exception to get a better error message.* 2. *Saving and
>>> retrieving experiment execution error details*. Earlier we were
>>> not persisting any execution error details of the workflow. 
>>> Error details are only available only if the developers are
>>> monitoring experiment at the time of error occurs. *This is not
>>> feasible if users would only want to go through the error
>>> details later without doing monitoring
>> from
>>> the beginning. Therefore we are now persisting the error
>>> details (not
>> as
>>> part of provenance data) of executing workflows (i.e.
>>> experiments) in
>> to
>>> the registry and also retrieving from registry through Airavata
>>> API (which calls the Registry API). These functions are
>>> available in the "ExecutionManager" of the AiravataAPI and
>>> "ProvenanceRegistry" in the RegistryAPI. * 3. Persisting &
>>> retrieving GFac job data. There are many benefits of saving the
>>> GRAM data, EC2 data etc. such as for debugging, analysis
>> etc.
>>> Currently these data are available only through the message 
>>> notifications. *It is not feasible for the same reasons
>>> mentioned for API change for saving error data. Hence the
>>> introduction of functions in the API to save & retrieve GFac
>>> job data.** These functions are available as 
>>> get/setApplicationJob***(...) functions in the
>>> "ProvenanceManager" of the AiravataAPI and "ProvenanceRegistry"
>>> in the RegistryAPI.*
>>> 
>> 
>> May I ask why we need setApplicationJob****() methods in the API
>> ? As far as I understood job information is populated by GFac and
>> other internal components. So why do we need API methods to set
>> those data ? (OR am I missing something ?)
>> 
> GFac and Internal components also use the AiravataAPI to populate
> the job data in the registry.
> 
> 
>> Thanks Amila
>> 
>> 
>>> 
>>> 
>>> Regards,
>>> 
>>> Saminda
>>> 
>>> 
>>> On Tue, May 21, 2013 at 4:59 PM, Raminder Singh <
>> raminderjsingh@gmail.com
>>>> wrote:
>>> 
>>>> Thanks Chathuri, I will test and let you know.
>>>> 
>>>> Raman On May 21, 2013, at 4:55 PM, Chathuri Wimalasena
>>>> wrote:
>>>> 
>>>>> Hi Raman,
>>>>> 
>>>>> The method Saminda mentioned will work for your case. I
>>>>> tested with
>>> your
>>>>> test class. I fixed deserialization issues from REST
>>>>> service side.
>> You
>>>> will
>>>>> probably have to update both server and client.
>>>>> 
>>>>> Regards, Chathuri
>>>>> 
>>>>> 
>>>>> On Tue, May 21, 2013 at 3:04 PM, Saminda Wijeratne <
>> samindaw@gmail.com
>>>>> wrote:
>>>>> 
>>>>>> So far what you are saying is true. If the client knows
>>>>>> which node
>>>> failed
>>>>>> he/she can get the list of errors for that node using
>>>>>> the getExecutionError(...) function. Use the "type" as
>>>>>> ALL and leave the gfacJobId as null. I couldn;t think of
>>>>>> doing this any other better
>> way
>>>> than
>>>>>> introducing a whole lot of functions which could be more
>>>> annoying/confusing
>>>>>> to figure-out. I'm welcome for suggestions for
>>>>>> improvements.
>>>>>> 
>>>>>> Saminda
>>>>>> 
>>>>>> 
>>>>>> On Tue, May 21, 2013 at 2:36 PM, Raminder Singh <
>>>> raminderjsingh@gmail.com
>>>>>>> wrote:
>>>>>> 
>>>>>>> Thanks Chathuri, the WorkflowExecutionError works.
>>>>>>> Integrating to
>>>> Error
>>>>>>> API brought few questions. We have ErrorTypes as :
>>>>>> WorkflowExecutionError,
>>>>>>> ExperimentExecutionError, NodeExecutionError,
>> GFacJobExecutionError,
>>>>>>> ExecutionError . How do we want clients to know which
>>>>>>> error type to
>>>>>> query?
>>>>>>> I ran a job and it failed, now i want to get error.
>>>>>>> Error occurred
>> at
>>>>>> GFac
>>>>>>> provider level and client does not have any
>>>>>>> information. Do we
>> expect
>>>>>>> client to query all different types?
>>>>>>> 
>>>>>>> I thing we can have is a method to get all error and
>>>>>>> return the
>> Error
>>>>>> type
>>>>>>> part of the object to help the client to classify the
>>>>>>> error
>> location.
>>>>>>> Other suggestions?
>>>>>>> 
>>>>>>> Raminder
>>>>>>> 
>>>>>>> On May 21, 2013, at 10:57 AM, Chathuri Wimalasena
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Hi Raman,
>>>>>>>> 
>>>>>>>> WorkflowExecutionError method not returning any data
>>>>>>>> issue should
>> be
>>>>>>> fixed
>>>>>>>> now. Can you get an update and check.
>>>>>>>> 
>>>>>>>> Regards, Chathuri
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Tue, May 21, 2013 at 10:02 AM, Raminder Singh 
>>>>>>>> <raminderjsingh@gmail.com>wrote:
>>>>>>>> 
>>>>>>>>> Thank, these are very useful for API users. Only
>>>>>>>>> thing i found
>>>>>> confusing
>>>>>>>>> for the user is API managers.  ProvenanceManager is
>>>>>>>>> the one used
>> to
>>>>>> get
>>>>>>>>> experiment status. Users will get FINISHED or
>>>>>>>>> FAILED status from
>>> the
>>>>>>> data
>>>>>>>>> object. On failure, i was trying to find a method
>>>>>>>>> to get the
>> error
>>>>>> data
>>>>>>> but
>>>>>>>>> i was not able to find as those methods exist in
>> ExecutionManager.
>>>>>>>>> According to me, experiment status and in case of
>>>>>>>>> failures get
>>> error
>>>>>>> need
>>>>>>>>> to be part of one manager. May be a
>>>>>>>>> ExperimentManager?  Getting
>>>> output
>>>>>>> data
>>>>>>>>> can be left as part of provenance manages as only
>>>>>>>>> few client may
>>> want
>>>>>> to
>>>>>>>>> get the data in few cases. Getting error detail in
>>>>>>>>> case of error
>> is
>>>>>>> obvious
>>>>>>>>> step.
>>>>>>>>> 
>>>>>>>>> Other thing can you please explain the difference
>>>>>>>>> between WorkflowExecutionError,
>>>>>>>>> ExperimentExecutionError,
>>> NodeExecutionError,
>>>>>>>>> GFacExecutionError on a Wiki page for the users.
>>>>>>>>> Also add a
>> method
>>> to
>>>>>>> get
>>>>>>>>> JobID as that is required to get
>>>>>>>>> GFacExecutionError.
>>>>>>>>> 
>>>>>>>>> I tested the WorkflowExecutionError method and its
>>>>>>>>> not returning
>>> any
>>>>>>> data.
>>>>>>>>> Can you please test that as well?
>>>>>>>>> 
>>>>>>>>> ExperimentData data =
>>>>>>>>> 
>> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
>>>>>>>>>
>> 
List<WorkflowExecutionError> experimentExecutionErrors =
>>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>>> 
>> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
>>>>>>>>>
>> 
experimentID);
>>>>>>>>> 
>>>>>>>>> Thanks Raminder
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On May 20, 2013, at 5:05 PM, Saminda Wijeratne
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Following updates will be present Airavata API
>>>>>>>>>> from 0.8 release.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 1. *Error message from the immediate exception
>>>>>>>>>> used as the error
>>>>>>>>> message
>>>>>>>>>> for the AiravataAPInvocationException*. Earlier
>>>>>>>>>> it was just
>> "Error
>>>>>>>>>> invoking API". *This change was made so that the
>>>>>>>>>> users of the
>> API
>>>>>> can
>>>>>>>>>> show the error from the exception without having
>>>>>>>>>> to dig in to
>> the
>>>>>>> inner
>>>>>>>>>> exception to get a better error message.* 2.
>>>>>>>>>> *Saving and retrieving experiment execution error
>>>>>>>>>> details*.
>>>>>> Earlier
>>>>>>>>>> we were not persisting any execution error
>>>>>>>>>> details of the
>>> workflow.
>>>>>>>>> Error
>>>>>>>>>> details are only available only if the developers
>>>>>>>>>> are monitoring
>>>>>>>>> experiment
>>>>>>>>>> at the time of error occurs. *This is not
>>>>>>>>>> feasible if users
>> would
>>>>>> only
>>>>>>>>>> want to go through the error details later
>>>>>>>>>> without doing
>>> monitoring
>>>>>>>>> from
>>>>>>>>>> the beginning. Therefore we are now persisting
>>>>>>>>>> the error details
>>>>>> (not
>>>>>>>>> as
>>>>>>>>>> part of provenance data) of executing workflows
>>>>>>>>>> (i.e.
>>> experiments)*
>>>>>> in
>>>>>>>>>> to the registry and also retrieving from registry
>>>>>>>>>> through
>> Airavata
>>>>>> API
>>>>>>>>>> (which calls the Registry API).
>>>>>>>>>> 
>>>>>>>>>> For now we have the functions for saving and
>>>>>>>>>> retrieving errors
>> in
>>>> the
>>>>>>>>>> ExecutionManager[1]. We need to decide if this is
>>>>>>>>>> the correct
>>> place
>>>>>> to
>>>>>>>>> put
>>>>>>>>>> these functions. Your thoughts in this matter are
>>>>>>>>>> most
>> welcome...
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Thanks, Saminda
>>>>>>>>>> 
>>>>>>>>>> 1.
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>>> 
>> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>>
>>>
>>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJRuyxqAAoJEOEgD2XReDo54nwIAKjDjVqPk6YldnJD+T91k2HU
Yj/VcvH/XAxvoZOu9zGpcUFNuLPbSiJEJes9Tx3Wk63k8v+l4rxTTRB5rWihbyYn
xPEKtSnD8sCAiuFXlRHWLygzE80OVhC9Q43xkrnjjubKaWKp3ZgU1uF1ZL6LiTZb
R83+Yl2WCdAAckX1W+567APk202nw93GLKGjmRcuj2mBPKgWXzzD7AX3cQX6pt9e
g93PKGWJWdk/kb6YlhjkEfKH2RxJS/AgZvUYSAmf1ELL+siGqxEEOg7bwWwpfLNd
NsjUr/rN88EcGfUKxYb4fSh1zGceqp02rWG2TwufZEIVaF8WjztY6CsE2TgfS0M=
=vpMl
-----END PGP SIGNATURE-----

Mime
View raw message