airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shameera Rathnayaka <shameerai...@gmail.com>
Subject Re: Airavata GSoC 2013 Master Project
Date Mon, 22 Apr 2013 20:29:35 GMT
Hi all,

Hangout session made it clear what we supposed to address with master
project.
As we all agreed JSON would be good replacement to existing XML messages.
The problem is how we are going replace XML with JSON. One solution is
replace the whole XML related works with JSON. Yes it is good if we don't
gain
considerable advantage using XML. But this solution need lot of changes to
Airavata server side. That would be a big project which is not possible in
this stage.
Then came up a solution that we are not going to replace the whole XML with
JSON
but send JSON request instead of SOAP(XML) request to the server(which
works with SOAP).
For this case we need a proper way to convert JSON <--> XML(SOAP).

In the hangout Subho proposed a way to do this without changing the
Airavata server side
 which is a good point.I think he will explain it here.

What i am suggesting is, we send the JSON message directly to Airavata
Backend(or Registry)
When the message gets build after the transport phase, convert JSON message
to SOAP(XML).
>From that point message will treated as SOAP message.

If we look at the JSON <--> XML conversion there are set of third party
libraries we
can use for. But before selecting a one we need to think about problems
having
with JSON <--> XML and how these libraries handle those issues. Because we
need a robust
way to do this conversions.

Please go through intruductions secssion of this paper[1] which nicely
explain the issues having with JSON <--> XML conversion. In the paper
itself provide an solution for this. But I have done the same thing only
using XMLSchema of the XML message(which is available with Airavata) as
part of my previous GSOC project. We can either use JXON or implement
simple solution with XMLSchema, for our implementations.


Thanks,
Shameera.

[1]
http://www.balisage.net/Proceedings/vol7/html/Lee01/BalisageVol7-Lee01.html#d58673e161


On Sat, Apr 20, 2013 at 3:51 AM, Suresh Marru <smarru@apache.org> wrote:

> Hi Lahiru,
>
> I am referring to GFac Descriptions. As we describe the inputs and
> outputs, the Service Map XML [1]  and an associated abstract WSDL is
> generated in the registered with Registry. For a while, lets ignore how
> these documents get to registry.
>
> The workflow  composer has to browse the registry, pull multiple of these
> registered application WSDl's, and allow user to drag drop. This is current
> way. I agree with all GSoC students that this needs to be changed to a
> light weight JSON documents.
>
> I am trying to clarify the same. But I think the point is not getting
> across fully and the focus is getting into how these documents get in and
> get out of registry or to workflow interpreter. Which as you know, quickly
> we can have Axis2 enable the REST EPR's. But the key is to get the document
> description format to be light weight.
>
> Suresh
> [1] -
> https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/gfac-schema/src/main/resources/schemas/ServiceDescription.xsd
>
>
> On Apr 19, 2013, at 3:31 PM, Lahiru Gunathilake <glahiru@gmail.com> wrote:
>
> > Hi Suresh,
> >
> > The issue is Airavata is using WSDL's for describing application
> > interfaces...
> >
> > I am not clear what you meant by this.
> >
> > Lahiru
> >
> >
> > On Fri, Apr 19, 2013 at 3:31 PM, Lahiru Gunathilake <glahiru@gmail.com
> >wrote:
> >
> >> Hi Suresh,
> >>
> >>
> >> On Fri, Apr 19, 2013 at 1:37 PM, Suresh Marru <smarru@apache.org>
> wrote:
> >>
> >>> Hi Shameera,
> >>>
> >>> Thank you for the detailed email. I will address every one since this
> is
> >>> a common misunderstanding.
> >>>
> >>> Hi All,
> >>>
> >>> We are not solving the problem of Airavata server reading SOAP
> messages.
> >>> Please totally ignore this fact. It is a 2 week project to move away
> from
> >>> Axis2 and replace it with light weight Jax-RS, thats a byproduct of the
> >>> current gsoc projects but not the focus.
> >>>
> >>> Forget all service to service communications - that is clearly
> >>> distracting.
> >>>
> >>> The key focus of Airavata GSoC Master project is to find a solution to
> >>> application description and workflow composition. Please pay attention
> to
> >>> the data model. Understand the GFac schema and how it is used for
> workflow
> >>> composition, binding workflow inputs. Second, understand how workflow
> >>> execution context is used in Airavata and propose an alternative.
> Propose
> >>> an alternative for these XML schemas. And how a web based XBaya can
> pull
> >>> this document (currently XML) and show a list of applications and allow
> >>> users to construct workfows with them. Once you have a workflow, a web
> >>> interface should be able to load up the workflow and allow user to
> find the
> >>> type of it and also provide metadata about each inputs.
> >>>
> >>> Summarizing again, the issues is not SOAP vs JSON. The issue is
> Airavata
> >>> is using WSDL's for describing application interfaces. A java script
> based
> >>> workflow composer reading these wsdl's will be heavy weight. So what
> is the
> >>> light weight alternative, and how do we accomplish it?
> >>>
> >> Suresh, can you please explain bit more what you meant by this
> statement ?
> >>
> >> Lahiru
> >>
> >>>
> >>> A good way to learn this is, register a command line application in
> >>> XBaya, and understanding what is being saved in the registry. How it is
> >>> saved is indeed through a RESTful Registry service. So please debug
> through
> >>> all these steps and get to bottom of this problem.
> >>>
> >>> Suresh
> >>>
> >>> On Apr 19, 2013, at 1:11 PM, Shameera Rathnayaka <
> shameerainfo@gmail.com>
> >>> wrote:
> >>>
> >>>> Hi Suresh and All,
> >>>>
> >>>> Web based UI (with JS), it is always good to use JSON instead of SOAP
> as
> >>>> JSON is pretty much handy and easy with JS than SOAP. Not only that,
> >>> when
> >>>> we consider the amount of data need to be transferred over network
> for a
> >>>> same message (native)JSON require less amount of data than SOAP. As
> >>> this is
> >>>> a considerable fact when we deal about network traffic therefore JSON
> >>> would
> >>>> be an idea solution.
> >>>>
> >>>> As Airavata backend has tuned to process SOAP messages(Axis2 which is
> >>> used
> >>>> by Airavata server, is a SOAP processing web engine[1]) when we send
a
> >>> JSON
> >>>> message to either server or registry it needs to be transferred to
> SOAP
> >>>> (which is a costly operation) to further process. Here i am proposing
> a
> >>>> technique rather than transfer JSON to SOAP in first hand, we keep the
> >>> JSON
> >>>> stream as it is and provide an interface (
> >>> XMLStreamReader/XMLStreamWriter)
> >>>> to get SOAP(XML) info-set while processing JSON Stream underneath. We
> >>> can
> >>>> use AXIOM[2] like XML infoset compliant object model implementation
> >>> which
> >>>> support on-demand building to read and write SOAP info-set on the fly.
> >>> This
> >>>> implementation inline with axis2 implementations too, If I am correct,
> >>>> current REST calls are sent with SOAP request body but with this kind
> of
> >>>> improvement we can send REST call to Registry with JSON, which is most
> >>>> prefer data exchange language with REST.
> >>>>
> >>>> I have implemented this kind of scenario to improve Apache Axis2 JSON
> >>>> support(my previous reply refer to that) as my previous GSOC project.
> >>> And
> >>>> it has good performance numbers than ADB(Axis2 Databinding).
> >>>>
> >>>> Either we can use currently available XML schema or introduce new JSON
> >>>> schema for this JSON messages(data structure of both schemas should
be
> >>>> exact same). It is good to have separate JSON schemas for JSON
> messages
> >>>> even it has maintenance issue as we need to keep both schemas in sync.
> >>> But
> >>>> as it is not encouraged to do schema changes this won't be an issue
at
> >>> all.
> >>>>
> >>>> Also Not only for Web base UI, we can further improve existing Java
> >>> client
> >>>> api to send JSON messages to the server by implementing API interfaces
> >>> for
> >>>> JSON data binding. With this either user can send SOAP or JSON
> messages
> >>> to
> >>>> Server and Registry. Both messages treated as same in server side.
> >>>>
> >>>> Here i am addressing only JSON message transfer and processing phase
> of
> >>> the
> >>>> master project. If everyone ok with this approach, I am very much
> happy
> >>> to
> >>>> submit a proposal for this. And I hope this implementation can
> perfectly
> >>>> match with GSOC scope too.
> >>>>
> >>>> Any suggestions or alternatives are well come.
> >>>>
> >>>>
> >>>> Thanks,
> >>>> Shameera.
> >>>>
> >>>>
> >>>> [1] http://axis.apache.org/axis2/java/core/
> >>>> [2] http://ws.apache.org/axiom/
> >>>>
> >>>>
> >>>> On Fri, Apr 19, 2013 at 3:55 PM, Suresh Marru <smarru@apache.org>
> >>> wrote:
> >>>>
> >>>>> Hi Vijayendra,
> >>>>>
> >>>>> Please refer to [1]. The POJO Client API I referred to is the client
> >>> which
> >>>>> talks to Registry. But the WSDL is about the application being
> >>> described. I
> >>>>> will revisit this document again to bring this out clearly.
> >>>>>
> >>>>> The SOAP/REST comes only on how a web or a stand alone java clients
> >>> talks
> >>>>> to Airavata Server. For simplification, I would assume all services
> >>>>> interactions as JDBC connections. Lets forget a java script is not
> good
> >>>>> directly with JDBC. I am trying to make you all focus on saying,
by
> >>> some
> >>>>> exchange you will know how to put and get data from registry. Lets
> >>> focus on
> >>>>> the data itself, and expressiveness. We can come back to the
> mechanics
> >>> of
> >>>>> interactions, after you fully understand the data models.
> >>>>>
> >>>>> So there is no POJO to WSDL conversion. The WSDL's in this context
> are
> >>> the
> >>>>> description of the nodes within the workflow. WSDL's are used for
> >>> these,
> >>>>> not for the SOAP reason, but since they envelop XML Schema of
> >>> input/outputs
> >>>>> (annotated with metadata). XBaya only looks at the application
> service
> >>>>> WSDL's and extracts out the inputs and their types.
> >>>>>
> >>>>> In the diagram in [1] ignore the lines, they can be SOAP, REST,
or
> >>>>> something else. Focus on what is going on over the wire in A1, A2
and
> >>> W1,
> >>>>> W2, W3.
> >>>>>
> >>>>> Suresh
> >>>>>
> >>>>> [1] - https://cwiki.apache.org/confluence/display/AIRAVATA/GSoC+2013
> >>>>> On Apr 18, 2013, at 6:21 PM, Vijayendra Grampurohit <
> >>>>> vijayendra.sdm@gmail.com> wrote:
> >>>>>
> >>>>>> Hi Suresh
> >>>>>>
> >>>>>> In the above conversation you wrote * - We have focused lately
on
> the
> >>>>> POJO
> >>>>>> Airavata Client API, the reason for not having a service layer
> around
> >>> is
> >>>>>> intentional. We wanted to first fully stabilize the API and
service
> >>>>> layers
> >>>>>> were distracting. More over the initial gateways integrating
with
> >>>>> Airavata
> >>>>>> were java based (with an exception of couple) so it worked well.
Now
> >>> is a
> >>>>>> good time to start exploring the service API.
> >>>>>>
> >>>>>> *
> >>>>>> Here I have a small doubt. you wrote POJO interacts with the
> Airavata
> >>>>>> client API(Registry) ? But the data that is stored in the Registry
> is
> >>>>> wsdl
> >>>>>> . So it means the data is converted from  POJO to WSDL while
storing
> >>> in
> >>>>>> Registry . Is this correct ?
> >>>>>>
> >>>>>>
> >>>>>> Regards
> >>>>>> Vijayendra
> >>>>>>
> >>>>>> *
> >>>>>> *
> >>>>>>
> >>>>>>
> >>>>>> On Wed, Apr 17, 2013 at 6:30 PM, Shameera Rathnayaka <
> >>>>> shameerainfo@gmail.com
> >>>>>>> wrote:
> >>>>>>
> >>>>>>> Hi devs,
> >>>>>>>
> >>>>>>> FYI, Current Apache Axis2 trunk has new feature which provide
XML
> >>>>> info-set
> >>>>>>> while processing JSON stream[1].( NOTE: Here it doesn't
convert
> JSON
> >>> to
> >>>>> XML
> >>>>>>> which is more cost effective). Even it implemented to Apache
Axis2
> >>> it is
> >>>>>>> not tightly couple with aixs2, we can use that for our own
> purpose. I
> >>>>> think
> >>>>>>> it would be a good solution for this.
> >>>>>>>
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Shameera.
> >>>>>>>
> >>>>>>> [1] https://issues.apache.org/jira/browse/AXIS2-5362
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Apr 17, 2013 at 4:02 PM, Sandeep Panem
> >>>>>>> <sandeep.panem870@gmail.com>wrote:
> >>>>>>>
> >>>>>>>> Instead of an interface which converts json to soap
and then
> sending
> >>>>> soap
> >>>>>>>> data to airavata services,we should follow standard
data format of
> >>> json
> >>>>>>>> which is compatible with all browsers and easily understandable
> and
> >>>>>>> reduces
> >>>>>>>> complexity.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Wed, Apr 17, 2013 at 2:13 AM, Mattmann, Chris A (398J)
<
> >>>>>>>> chris.a.mattmann@jpl.nasa.gov> wrote:
> >>>>>>>>
> >>>>>>>>> Hey Guys,
> >>>>>>>>>
> >>>>>>>>> Best way to find out is to email legal-discuss@apache.org.
> >>>>>>>>>
> >>>>>>>>> I'll try and look at it either there or here and
comment.
> >>>>>>>>>
> >>>>>>>>> Cheers,
> >>>>>>>>> Chris
> >>>>>>>>>
> >>>>>>>>>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>>>>>>> Chris Mattmann, Ph.D.
> >>>>>>>>> Senior Computer Scientist
> >>>>>>>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109
USA
> >>>>>>>>> Office: 171-266B, Mailstop: 171-246
> >>>>>>>>> Email: chris.a.mattmann@nasa.gov
> >>>>>>>>> WWW:  http://sunset.usc.edu/~mattmann/
> >>>>>>>>>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>>>>>>> Adjunct Assistant Professor, Computer Science Department
> >>>>>>>>> University of Southern California, Los Angeles,
CA 90089 USA
> >>>>>>>>>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> -----Original Message-----
> >>>>>>>>> From: Marlon Pierce <marpierc@iu.edu>
> >>>>>>>>> Reply-To: "dev@airavata.apache.org" <dev@airavata.apache.org>
> >>>>>>>>> Date: Tuesday, April 16, 2013 1:15 PM
> >>>>>>>>> To: "dev@airavata.apache.org" <dev@airavata.apache.org>
> >>>>>>>>> Subject: Re: Airavata GSoC 2013 Master Project
> >>>>>>>>>
> >>>>>>>>>> -----BEGIN PGP SIGNED MESSAGE-----
> >>>>>>>>>> Hash: SHA1
> >>>>>>>>>>
> >>>>>>>>>> That license doesn't look too bad, but I don't
know how strict
> we
> >>>>> need
> >>>>>>>>>> to be.  It is possible that the author, if contacted,
will adopt
> >>> an
> >>>>>>>>>> Apache or Apache-compatible license.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Marlon
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On 4/16/13 4:12 PM, Subho Banerjee wrote:
> >>>>>>>>>>>> If we were two pursue JSON for descriptions
applications and
> >>>>>>>>>>>> workflow, we need to explore validations.
I wonder if there
> are
> >>>>>>>>>>>> any good implementations of the json-schema
validation spec -
> >>>>>>>>>>>>
> >>> http://tools.ietf.org/pdf/draft-fge-json-schema-validation-00.pdf
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> From
> >>>>>>>>>>>>
> >>>>>>>>>> what I know, the best (open and closest to implementing
the
> entire
> >>>>>>>>>>> spec) validator for JSON schemas is JSV[1].
I am not
> particularly
> >>>>>>>>>>> sure if this license[2] is compatible to
the Apache license. If
> >>> it
> >>>>>>>>>>> is, then using this could be one way to
go ahead. However, in
> the
> >>>>>>>>>>> case that it is not compatible, then we
might have to fork some
> >>>>>>>>>>> sort of a half baked validator and modify
it to suit the needs
> of
> >>>>>>>>>>> the project. I will look into this and let
you know what I
> find.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> [1] - https://github.com/garycourt/JSV [2]
-
> >>>>>>>>>>> https://github.com/garycourt/JSV/blob/master/README.md#license
> >>>>>>>>>>>
> >>>>>>>>>> -----BEGIN PGP SIGNATURE-----
> >>>>>>>>>> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
> >>>>>>>>>> Comment: GPGTools - http://gpgtools.org
> >>>>>>>>>> Comment: Using GnuPG with Thunderbird -
> http://www.enigmail.net/
> >>>>>>>>>>
> >>>>>>>>>> iQEcBAEBAgAGBQJRbbFtAAoJEOEgD2XReDo5wRoIAIhK2u0GL+lp2TL6jPJbHJ/U
> >>>>>>>>>> W+lVr69MAhPdfiMUxqNjHZwJSjn0yU5b310/8rqQ5RMOw6BUHxAaKo3Iinhfq2xf
> >>>>>>>>>> DPyzhGrawiVuUXvkco6aUI4QmHqXl+7PpXC8Sr0o6FuATDNX7DcOrdlSK6Gp2MXi
> >>>>>>>>>> BrREBZ4LbY//yue5LkhKkGuz6q7tdBp94AQPrBnbtmTuI81uVBxCLOb0DUxkhOTE
> >>>>>>>>>> 87PgkhyWDyR+2wLwVze2Ng1EaUmADcElhRZvBtLqESFg2LoN4NDIzhBX/XdvxKaf
> >>>>>>>>>> vlgnmIZS+Uc7ftAijYuQPg4246RJKLDQYFr5Ms3tomu1vt4BHArNuMQdGzoflPs=
> >>>>>>>>>> =bQv0
> >>>>>>>>>> -----END PGP SIGNATURE-----
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> -Sandeep Panem
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Best Regards,
> >>>>>>> Shameera Rathnayaka.
> >>>>>>>
> >>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Best Regards,
> >>>> Shameera Rathnayaka.
> >>>>
> >>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>
> >>>
> >>
> >>
> >> --
> >> System Analyst Programmer
> >> PTI Lab
> >> Indiana University
> >>
> >
> >
> >
> > --
> > System Analyst Programmer
> > PTI Lab
> > Indiana University
>
>


-- 
Best Regards,
Shameera Rathnayaka.

Blog : http://shameerarathnayaka.blogspot.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message