airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suresh Marru <sma...@apache.org>
Subject Re: Airavata GSoC 2013 Master Project
Date Fri, 19 Apr 2013 22:21:33 GMT
Hi Lahiru,

I am referring to GFac Descriptions. As we describe the inputs and outputs, the Service Map
XML [1]  and an associated abstract WSDL is generated in the registered with Registry. For
a while, lets ignore how these documents get to registry.

The workflow  composer has to browse the registry, pull multiple of these registered application
WSDl's, and allow user to drag drop. This is current way. I agree with all GSoC students that
this needs to be changed to a light weight JSON documents.

I am trying to clarify the same. But I think the point is not getting across fully and the
focus is getting into how these documents get in and get out of registry or to workflow interpreter.
Which as you know, quickly we can have Axis2 enable the REST EPR's. But the key is to get
the document description format to be light weight. 

Suresh
[1] - https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/gfac-schema/src/main/resources/schemas/ServiceDescription.xsd


On Apr 19, 2013, at 3:31 PM, Lahiru Gunathilake <glahiru@gmail.com> wrote:

> Hi Suresh,
> 
> The issue is Airavata is using WSDL's for describing application
> interfaces...
> 
> I am not clear what you meant by this.
> 
> Lahiru
> 
> 
> On Fri, Apr 19, 2013 at 3:31 PM, Lahiru Gunathilake <glahiru@gmail.com>wrote:
> 
>> Hi Suresh,
>> 
>> 
>> On Fri, Apr 19, 2013 at 1:37 PM, Suresh Marru <smarru@apache.org> wrote:
>> 
>>> Hi Shameera,
>>> 
>>> Thank you for the detailed email. I will address every one since this is
>>> a common misunderstanding.
>>> 
>>> Hi All,
>>> 
>>> We are not solving the problem of Airavata server reading SOAP messages.
>>> Please totally ignore this fact. It is a 2 week project to move away from
>>> Axis2 and replace it with light weight Jax-RS, thats a byproduct of the
>>> current gsoc projects but not the focus.
>>> 
>>> Forget all service to service communications - that is clearly
>>> distracting.
>>> 
>>> The key focus of Airavata GSoC Master project is to find a solution to
>>> application description and workflow composition. Please pay attention to
>>> the data model. Understand the GFac schema and how it is used for workflow
>>> composition, binding workflow inputs. Second, understand how workflow
>>> execution context is used in Airavata and propose an alternative. Propose
>>> an alternative for these XML schemas. And how a web based XBaya can pull
>>> this document (currently XML) and show a list of applications and allow
>>> users to construct workfows with them. Once you have a workflow, a web
>>> interface should be able to load up the workflow and allow user to find the
>>> type of it and also provide metadata about each inputs.
>>> 
>>> Summarizing again, the issues is not SOAP vs JSON. The issue is Airavata
>>> is using WSDL's for describing application interfaces. A java script based
>>> workflow composer reading these wsdl's will be heavy weight. So what is the
>>> light weight alternative, and how do we accomplish it?
>>> 
>> Suresh, can you please explain bit more what you meant by this statement ?
>> 
>> Lahiru
>> 
>>> 
>>> A good way to learn this is, register a command line application in
>>> XBaya, and understanding what is being saved in the registry. How it is
>>> saved is indeed through a RESTful Registry service. So please debug through
>>> all these steps and get to bottom of this problem.
>>> 
>>> Suresh
>>> 
>>> On Apr 19, 2013, at 1:11 PM, Shameera Rathnayaka <shameerainfo@gmail.com>
>>> wrote:
>>> 
>>>> Hi Suresh and All,
>>>> 
>>>> Web based UI (with JS), it is always good to use JSON instead of SOAP as
>>>> JSON is pretty much handy and easy with JS than SOAP. Not only that,
>>> when
>>>> we consider the amount of data need to be transferred over network for a
>>>> same message (native)JSON require less amount of data than SOAP. As
>>> this is
>>>> a considerable fact when we deal about network traffic therefore JSON
>>> would
>>>> be an idea solution.
>>>> 
>>>> As Airavata backend has tuned to process SOAP messages(Axis2 which is
>>> used
>>>> by Airavata server, is a SOAP processing web engine[1]) when we send a
>>> JSON
>>>> message to either server or registry it needs to be transferred to SOAP
>>>> (which is a costly operation) to further process. Here i am proposing a
>>>> technique rather than transfer JSON to SOAP in first hand, we keep the
>>> JSON
>>>> stream as it is and provide an interface (
>>> XMLStreamReader/XMLStreamWriter)
>>>> to get SOAP(XML) info-set while processing JSON Stream underneath. We
>>> can
>>>> use AXIOM[2] like XML infoset compliant object model implementation
>>> which
>>>> support on-demand building to read and write SOAP info-set on the fly.
>>> This
>>>> implementation inline with axis2 implementations too, If I am correct,
>>>> current REST calls are sent with SOAP request body but with this kind of
>>>> improvement we can send REST call to Registry with JSON, which is most
>>>> prefer data exchange language with REST.
>>>> 
>>>> I have implemented this kind of scenario to improve Apache Axis2 JSON
>>>> support(my previous reply refer to that) as my previous GSOC project.
>>> And
>>>> it has good performance numbers than ADB(Axis2 Databinding).
>>>> 
>>>> Either we can use currently available XML schema or introduce new JSON
>>>> schema for this JSON messages(data structure of both schemas should be
>>>> exact same). It is good to have separate JSON schemas for JSON messages
>>>> even it has maintenance issue as we need to keep both schemas in sync.
>>> But
>>>> as it is not encouraged to do schema changes this won't be an issue at
>>> all.
>>>> 
>>>> Also Not only for Web base UI, we can further improve existing Java
>>> client
>>>> api to send JSON messages to the server by implementing API interfaces
>>> for
>>>> JSON data binding. With this either user can send SOAP or JSON messages
>>> to
>>>> Server and Registry. Both messages treated as same in server side.
>>>> 
>>>> Here i am addressing only JSON message transfer and processing phase of
>>> the
>>>> master project. If everyone ok with this approach, I am very much happy
>>> to
>>>> submit a proposal for this. And I hope this implementation can perfectly
>>>> match with GSOC scope too.
>>>> 
>>>> Any suggestions or alternatives are well come.
>>>> 
>>>> 
>>>> Thanks,
>>>> Shameera.
>>>> 
>>>> 
>>>> [1] http://axis.apache.org/axis2/java/core/
>>>> [2] http://ws.apache.org/axiom/
>>>> 
>>>> 
>>>> On Fri, Apr 19, 2013 at 3:55 PM, Suresh Marru <smarru@apache.org>
>>> wrote:
>>>> 
>>>>> Hi Vijayendra,
>>>>> 
>>>>> Please refer to [1]. The POJO Client API I referred to is the client
>>> which
>>>>> talks to Registry. But the WSDL is about the application being
>>> described. I
>>>>> will revisit this document again to bring this out clearly.
>>>>> 
>>>>> The SOAP/REST comes only on how a web or a stand alone java clients
>>> talks
>>>>> to Airavata Server. For simplification, I would assume all services
>>>>> interactions as JDBC connections. Lets forget a java script is not good
>>>>> directly with JDBC. I am trying to make you all focus on saying, by
>>> some
>>>>> exchange you will know how to put and get data from registry. Lets
>>> focus on
>>>>> the data itself, and expressiveness. We can come back to the mechanics
>>> of
>>>>> interactions, after you fully understand the data models.
>>>>> 
>>>>> So there is no POJO to WSDL conversion. The WSDL's in this context are
>>> the
>>>>> description of the nodes within the workflow. WSDL's are used for
>>> these,
>>>>> not for the SOAP reason, but since they envelop XML Schema of
>>> input/outputs
>>>>> (annotated with metadata). XBaya only looks at the application service
>>>>> WSDL's and extracts out the inputs and their types.
>>>>> 
>>>>> In the diagram in [1] ignore the lines, they can be SOAP, REST, or
>>>>> something else. Focus on what is going on over the wire in A1, A2 and
>>> W1,
>>>>> W2, W3.
>>>>> 
>>>>> Suresh
>>>>> 
>>>>> [1] - https://cwiki.apache.org/confluence/display/AIRAVATA/GSoC+2013
>>>>> On Apr 18, 2013, at 6:21 PM, Vijayendra Grampurohit <
>>>>> vijayendra.sdm@gmail.com> wrote:
>>>>> 
>>>>>> Hi Suresh
>>>>>> 
>>>>>> In the above conversation you wrote * - We have focused lately on
the
>>>>> POJO
>>>>>> Airavata Client API, the reason for not having a service layer around
>>> is
>>>>>> intentional. We wanted to first fully stabilize the API and service
>>>>> layers
>>>>>> were distracting. More over the initial gateways integrating with
>>>>> Airavata
>>>>>> were java based (with an exception of couple) so it worked well.
Now
>>> is a
>>>>>> good time to start exploring the service API.
>>>>>> 
>>>>>> *
>>>>>> Here I have a small doubt. you wrote POJO interacts with the Airavata
>>>>>> client API(Registry) ? But the data that is stored in the Registry
is
>>>>> wsdl
>>>>>> . So it means the data is converted from  POJO to WSDL while storing
>>> in
>>>>>> Registry . Is this correct ?
>>>>>> 
>>>>>> 
>>>>>> Regards
>>>>>> Vijayendra
>>>>>> 
>>>>>> *
>>>>>> *
>>>>>> 
>>>>>> 
>>>>>> On Wed, Apr 17, 2013 at 6:30 PM, Shameera Rathnayaka <
>>>>> shameerainfo@gmail.com
>>>>>>> wrote:
>>>>>> 
>>>>>>> Hi devs,
>>>>>>> 
>>>>>>> FYI, Current Apache Axis2 trunk has new feature which provide
XML
>>>>> info-set
>>>>>>> while processing JSON stream[1].( NOTE: Here it doesn't convert
JSON
>>> to
>>>>> XML
>>>>>>> which is more cost effective). Even it implemented to Apache
Axis2
>>> it is
>>>>>>> not tightly couple with aixs2, we can use that for our own purpose.
I
>>>>> think
>>>>>>> it would be a good solution for this.
>>>>>>> 
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Shameera.
>>>>>>> 
>>>>>>> [1] https://issues.apache.org/jira/browse/AXIS2-5362
>>>>>>> 
>>>>>>> 
>>>>>>> On Wed, Apr 17, 2013 at 4:02 PM, Sandeep Panem
>>>>>>> <sandeep.panem870@gmail.com>wrote:
>>>>>>> 
>>>>>>>> Instead of an interface which converts json to soap and then
sending
>>>>> soap
>>>>>>>> data to airavata services,we should follow standard data
format of
>>> json
>>>>>>>> which is compatible with all browsers and easily understandable
and
>>>>>>> reduces
>>>>>>>> complexity.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Wed, Apr 17, 2013 at 2:13 AM, Mattmann, Chris A (398J)
<
>>>>>>>> chris.a.mattmann@jpl.nasa.gov> wrote:
>>>>>>>> 
>>>>>>>>> Hey Guys,
>>>>>>>>> 
>>>>>>>>> Best way to find out is to email legal-discuss@apache.org.
>>>>>>>>> 
>>>>>>>>> I'll try and look at it either there or here and comment.
>>>>>>>>> 
>>>>>>>>> Cheers,
>>>>>>>>> Chris
>>>>>>>>> 
>>>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>>>> Chris Mattmann, Ph.D.
>>>>>>>>> Senior Computer Scientist
>>>>>>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>>>>>>>> Office: 171-266B, Mailstop: 171-246
>>>>>>>>> Email: chris.a.mattmann@nasa.gov
>>>>>>>>> WWW:  http://sunset.usc.edu/~mattmann/
>>>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>>>> Adjunct Assistant Professor, Computer Science Department
>>>>>>>>> University of Southern California, Los Angeles, CA 90089
USA
>>>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Marlon Pierce <marpierc@iu.edu>
>>>>>>>>> Reply-To: "dev@airavata.apache.org" <dev@airavata.apache.org>
>>>>>>>>> Date: Tuesday, April 16, 2013 1:15 PM
>>>>>>>>> To: "dev@airavata.apache.org" <dev@airavata.apache.org>
>>>>>>>>> Subject: Re: Airavata GSoC 2013 Master Project
>>>>>>>>> 
>>>>>>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>>>>>>> Hash: SHA1
>>>>>>>>>> 
>>>>>>>>>> That license doesn't look too bad, but I don't know
how strict we
>>>>> need
>>>>>>>>>> to be.  It is possible that the author, if contacted,
will adopt
>>> an
>>>>>>>>>> Apache or Apache-compatible license.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Marlon
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On 4/16/13 4:12 PM, Subho Banerjee wrote:
>>>>>>>>>>>> If we were two pursue JSON for descriptions
applications and
>>>>>>>>>>>> workflow, we need to explore validations.
I wonder if there are
>>>>>>>>>>>> any good implementations of the json-schema
validation spec -
>>>>>>>>>>>> 
>>> http://tools.ietf.org/pdf/draft-fge-json-schema-validation-00.pdf
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> From
>>>>>>>>>>>> 
>>>>>>>>>> what I know, the best (open and closest to implementing
the entire
>>>>>>>>>>> spec) validator for JSON schemas is JSV[1]. I
am not particularly
>>>>>>>>>>> sure if this license[2] is compatible to the
Apache license. If
>>> it
>>>>>>>>>>> is, then using this could be one way to go ahead.
However, in the
>>>>>>>>>>> case that it is not compatible, then we might
have to fork some
>>>>>>>>>>> sort of a half baked validator and modify it
to suit the needs of
>>>>>>>>>>> the project. I will look into this and let you
know what I find.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> [1] - https://github.com/garycourt/JSV [2] -
>>>>>>>>>>> https://github.com/garycourt/JSV/blob/master/README.md#license
>>>>>>>>>>> 
>>>>>>>>>> -----BEGIN PGP SIGNATURE-----
>>>>>>>>>> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
>>>>>>>>>> Comment: GPGTools - http://gpgtools.org
>>>>>>>>>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>>>>>>>>>> 
>>>>>>>>>> iQEcBAEBAgAGBQJRbbFtAAoJEOEgD2XReDo5wRoIAIhK2u0GL+lp2TL6jPJbHJ/U
>>>>>>>>>> W+lVr69MAhPdfiMUxqNjHZwJSjn0yU5b310/8rqQ5RMOw6BUHxAaKo3Iinhfq2xf
>>>>>>>>>> DPyzhGrawiVuUXvkco6aUI4QmHqXl+7PpXC8Sr0o6FuATDNX7DcOrdlSK6Gp2MXi
>>>>>>>>>> BrREBZ4LbY//yue5LkhKkGuz6q7tdBp94AQPrBnbtmTuI81uVBxCLOb0DUxkhOTE
>>>>>>>>>> 87PgkhyWDyR+2wLwVze2Ng1EaUmADcElhRZvBtLqESFg2LoN4NDIzhBX/XdvxKaf
>>>>>>>>>> vlgnmIZS+Uc7ftAijYuQPg4246RJKLDQYFr5Ms3tomu1vt4BHArNuMQdGzoflPs=
>>>>>>>>>> =bQv0
>>>>>>>>>> -----END PGP SIGNATURE-----
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> -Sandeep Panem
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> Best Regards,
>>>>>>> Shameera Rathnayaka.
>>>>>>> 
>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> --
>>>> Best Regards,
>>>> Shameera Rathnayaka.
>>>> 
>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>> 
>>> 
>> 
>> 
>> --
>> System Analyst Programmer
>> PTI Lab
>> Indiana University
>> 
> 
> 
> 
> -- 
> System Analyst Programmer
> PTI Lab
> Indiana University


Mime
View raw message