axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Gainty <mgai...@hotmail.com>
Subject RE: [GSoC 2012] XML to JSON convention using XMLStreamWriter API and Schemas with google-gson
Date Sun, 12 Aug 2012 02:06:05 GMT

It seems we lose alot of meta data by casting off the SOAP 1.1 superstructure a partial list
of what we lose would include:
ability to configure to differing styles (JIBX-RI vs ADB vs XMLBeans)
ability to adhere to different protocols (SMTP and or FTP and or HTTP 1.1)
Ability to incorporate differing XSD's
Parser for both the Input and Output Message Receivers (MEP) goes away

I would like to know what advantages people see with using strictly JSON over a SOAP formatted
sender and receiver

thanks!
Martin 
______________________________________________ 
Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité

Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfaenger sein, so bitten
wir hoeflich um eine Mitteilung. Jede unbefugte Weiterleitung oder Fertigung einer Kopie ist
unzulaessig. Diese Nachricht dient lediglich dem Austausch von Informationen und entfaltet
keine rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von E-Mails koennen
wir keine Haftung fuer den Inhalt uebernehmen.
Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas le destinataire
prévu, nous te demandons avec bonté que pour satisfaire informez l'expéditeur. N'importe
quelle diffusion non autorisée ou la copie de ceci est interdite. Ce message sert à l'information
seulement et n'aura pas n'importe quel effet légalement obligatoire. Étant donné que les
email peuvent facilement être sujets à la manipulation, nous ne pouvons accepter aucune
responsabilité pour le contenu fourni.


> From: sagara.gunathunga@gmail.com
> Date: Sat, 11 Aug 2012 19:38:16 +0530
> Subject: Re: [GSoC 2012] XML to JSON convention using XMLStreamWriter API and Schemas
with google-gson
> To: java-dev@axis.apache.org
> 
> It seems your 1st approach provides much better performance than 2nd
> approach, can you incorporate improvements that I mentioned here[1] ?
> I believe applying those modification will solve performance issue
> with 2nd approach.
> 
> [1] - http://markmail.org/message/xyfxzn2clc3r5zfg
> 
> Thanks !
> 
> On Sat, Aug 11, 2012 at 4:07 AM, Shameera Rathnayaka
> <shameerainfo@gmail.com> wrote:
> > Hi devs,
> >
> > I did performance test with 1. SOAP support , 2 JSON Badgerfish , 3 JSON
> > first approach(pure json) and 4. JSON second
> > approach(XMLStreamReader/XMLStreamWriter implementation). Here i have
> > summarize the results i got.
> > I used same POJO service and send the exact same request in relevent
> > convention.
> >
> > Environment:
> > OS : Ubuntu 11.04
> > JDK 1.6_30
> > memory 4 GB
> >
> >                                         SOAP         JSON badgerfish
> > JSON 1st approach   JSON 2nd approach
> >
> > Concurrency Level:           50                   50
> > 50                               50
> > Time taken for tests:         243.13 sec       287.18 sec
> > 118.99                          200.0
> > Complete requests:          500000             500000
> > 500000                        500000
> > Total transferred:              2.2 GB             34.42 GB
> > 0.61 GB                       0.63 GB
> > Requests per second:      2056.53 #/sec   1741.04 #/sec             4201.69
> > #/sec               2499.91 #/sec
> > Time per request:             24.3 ms           28.7 ms
> > 11.90 ms                      20.00 ms
> > Time per request:(mean)  0.486 ms          0.544 ms                    0.238
> > ms                      0.4 ms
> >
> > Thanks,
> > Shameera.
> >
> >
> > On Wed, Aug 8, 2012 at 12:17 PM, Shameera Rathnayaka
> > <shameerainfo@gmail.com> wrote:
> >>
> >> HI devs,
> >>
> >> I finished the implementation and did a performance test with existing
> >> SOAP and implemented JSON. Here is the summary of the test results.
> >> For this i used same service and send the same request in both SOAP and
> >> JSON ways, size of JSON message is lower than SOAP message but requests are
> >> exactly same.
> >>
> >> Environment :
> >> OS - Ubuntu 11.10
> >> Axis2 1.7(current trunk)
> >> Tool - Java benchmark
> >>
> >> num of requests : 50000
> >> concurrency level: 10 (Total num of requests are 50000*10 = 500000 )
> >> Time taken for test :  with SOAP  178 sec and with JSON 140 sec
> >> Total transferred : with SOAP 2371000000 bytes and with JSON 674000000
> >> bytes
> >> Requests per second: with SOAP  2,801.76 [#/sec] (mean)  and with JSON
> >> 3,568.89 [#/sec] (mean)
> >>
> >> will do a performance test with concurrency level 50 and num of requests
> >> 500000 and give the summary of results.
> >>
> >> Thanks,
> >> Shameera.
> >>
> >>
> >>
> >>
> >> On Tue, Jul 31, 2012 at 10:50 AM, Shameera Rathnayaka
> >> <shameerainfo@gmail.com> wrote:
> >>>
> >>> HI devs,
> >>>
> >>> I have implemented XMLStreamWriter as well as XMLStreamReader to convert
> >>> XML <----> JSON with XMLSchema,
> >>> Basically it works fine, but still need few test to be done after that I
> >>> am willing to write performance test with implemented JSON support with
> >>> existing SOAP support.
> >>>
> >>> Thanks,
> >>> Shameera.
> >>>
> >>>
> >>>
> >>> On Mon, Jul 16, 2012 at 10:43 AM, Amila Suriarachchi
> >>> <amilasuriarachchi@gmail.com> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On Sun, Jul 15, 2012 at 10:53 PM, Shameera Rathnayaka
> >>>> <shameerainfo@gmail.com> wrote:
> >>>>>
> >>>>> Hi devs,
> >>>>>
> >>>>> I implemented this feature completely to convert XML to JSON with
the
> >>>>> help of XmlSchema of that XML document. I tested this with different
> >>>>> Schemas, it is working fine. I implemented this as a separate project,
so i
> >>>>> will integrate this with axis2 trunk and try with actual scenario.
> >>>>
> >>>>
> >>>> Can you use this technique at the reader level as well to support
> >>>> namespaces?
> >>>>
> >>>> Then do a performance comparison using a POJO service with the exiting
> >>>> json support and based the new gson based XMLstream API. According to
the
> >>>> result we need to optimise the later to have better performance.
> >>>>
> >>>> thanks,
> >>>> Amila.
> >>>>
> >>>>>
> >>>>>
> >>>>> Yes, This may be a standard way to convert XML to JSON conversion
as
> >>>>> other implementations don't convert JSON message correctly when
it has one
> >>>>> value array, unless implementation supports special XML structure
where
> >>>>> Json-lib does.
> >>>>>
> >>>>> Definitely we can improve the performance of this implementation.
May
> >>>>> be in the way that Sagara had suggested in his reply or in another
way.
> >>>>>
> >>>>> Thanks,
> >>>>> Shameera.
> >>>>>
> >>>>> On Sun, Jul 15, 2012 at 6:56 PM, Amila Suriarachchi
> >>>>> <amilasuriarachchi@gmail.com> wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Sun, Jul 15, 2012 at 2:19 PM, Sagara Gunathunga
> >>>>>> <sagara.gunathunga@gmail.com> wrote:
> >>>>>>>
> >>>>>>> On Sun, Jul 15, 2012 at 1:27 PM, Amila Suriarachchi
> >>>>>>> <amilasuriarachchi@gmail.com> wrote:
> >>>>>>> >
> >>>>>>> >
> >>>>>>> > On Sat, Jul 14, 2012 at 1:14 PM, Sagara Gunathunga
> >>>>>>> > <sagara.gunathunga@gmail.com> wrote:
> >>>>>>> >>
> >>>>>>> >> On Sat, Jul 14, 2012 at 12:42 PM, Shameera Rathnayaka
> >>>>>>> >> <shameerainfo@gmail.com> wrote:
> >>>>>>> >> > Hi devs,
> >>>>>>> >> >
> >>>>>>> >> > As Amila(Project mentor) suggested in previous
thread[0] , I
> >>>>>>> >> > have
> >>>>>>> >> > implemented
> >>>>>>> >> > XMLStreamWriter API to use schema of the processing
xml and gson
> >>>>>>> >> > together
> >>>>>>> >> > to convert XML ---> JSON. I have made my
progress up to support
> >>>>>>> >> > xml
> >>>>>>> >> > which
> >>>>>>> >> > don't have complex type elements with maxOccur
>1,  but may have
> >>>>>>> >> > simple
> >>>>>>> >> > type elements with maxOccur >=1 . That
means In the JSON point
> >>>>>>> >> > of view,
> >>>>>>> >> > my
> >>>>>>> >> > present implementation supports XML -->
JSON where there is not
> >>>>>>> >> > any JSON
> >>>>>>> >> > Arrays which has JSON objects in the converted
JSON String.
> >>>>>>> >> >
> >>>>>>> >> > Here is a summary of the implementation.
> >>>>>>> >> >
> >>>>>>> >> > GsonXmlStreamWriter which implements XMLStreamWriter
has a
> >>>>>>> >> > constructor
> >>>>>>> >> > to get JsonWriter and XmlSchema object. Then
it process
> >>>>>>> >> > XmlSchema and
> >>>>>>> >>
> >>>>>>> >> Are you plan to process  XmlSchema object with
every response ? If
> >>>>>>> >> so
> >>>>>>> >> processing  XmlSchema object will add some overhead
and result
> >>>>>>> >> into
> >>>>>>> >> performance.  Axis2 already has JSON support main
objective of
> >>>>>>> >> this
> >>>>>>> >> project is to provide a high performance JSON implementation,
> >>>>>>> >> processing  XmlSchema object with every response
does not help to
> >>>>>>> >> meet
> >>>>>>> >> your expectations. Instead I suggest to process
 XmlSchema object
> >>>>>>> >> only
> >>>>>>> >> one-time during the deployment after the AxisService
creation and
> >>>>>>> >> store result of  XmlSchema processing  with AxisService
object as
> >>>>>>> >> a
> >>>>>>> >> light-weight object ( Let's say a map)   and use
this light weight
> >>>>>>> >> map
> >>>>>>> >> with response. In this approach overhead if relatively
low.
> >>>>>>> >
> >>>>>>> >
> >>>>>>> > This is a POC level approach to convert xml stream
to json using
> >>>>>>> > xml schema.
> >>>>>>> > For an example there are some techniques like
> >>>>>>> > bagger fish and others to address this problem. But
those things
> >>>>>>> > complicated
> >>>>>>> > the produced json stream.
> >>>>>>> >
> >>>>>>> > Once it proves at the POC level we can think of improving
> >>>>>>> > performance.
> >>>>>>> >
> >>>>>>> > If you look at the current axis2 json support does
that properly
> >>>>>>> > address the
> >>>>>>> > problem shameera mentioned earlier? i.e serialising
the
> >>>>>>> > array elements when there is on element. And also can
that be used
> >>>>>>> > with some
> >>>>>>> > thing like ADB or WSO2 Data Services,rules etc..
> >>>>>>> > Idea here to develop a technique to address those problems
properly
> >>>>>>> > while
> >>>>>>> > converting an xml stream to json stream.
> >>>>>>>
> >>>>>>> I don't have any issue with the technique proposed here
(i.e.
> >>>>>>> processing XMLSchema to identify structure ). My only concern
is why
> >>>>>>> we should process XMLSchema with every response because
in each and
> >>>>>>> every response we end up getting same result, why shouldn't
we
> >>>>>>> process
> >>>>>>> XMLSchema only one time and reuse results ? May be as last
step of
> >>>>>>> deployment or with very first response processing ?
> >>>>>>>
> >>>>>>> Of course we can keep it as future improvement for the moment
but
> >>>>>>> Axis2 having good history of forgetting those improvement
 after
> >>>>>>> initial development every time when I dig into code I found
couple of
> >>>>>>> such TODOs.
> >>>>>>
> >>>>>>
> >>>>>> :) I am not telling to do it after GSoc project. But first we
need to
> >>>>>> have a proper POC.
> >>>>>>
> >>>>>> thanks,
> >>>>>> Amila.
> >>>>>>>
> >>>>>>>
> >>>>>>> Thanks !
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> >
> >>>>>>> > thanks,
> >>>>>>> > Amila.
> >>>>>>> >
> >>>>>>> >>
> >>>>>>> >>
> >>>>>>> >> Thanks !
> >>>>>>> >>
> >>>>>>> >> > use a queue to keep the schema structure.
At the moment i have
> >>>>>>> >> > only
> >>>>>>> >> > implemented
> >>>>>>> >> > writeStartElement(String localName) , writeCharaters(String
> >>>>>>> >> > text) ,
> >>>>>>> >> > writeEndElement()
> >>>>>>> >> > and writeEndDocument() functions. I am processing
the queue
> >>>>>>> >> > which has
> >>>>>>> >> > the schema structure and put it to another
stack to identify the
> >>>>>>> >> > scope
> >>>>>>> >> > of
> >>>>>>> >> > end elements.
> >>>>>>> >> >
> >>>>>>> >> > I am now working on, improving this to support
xml which has
> >>>>>>> >> > complex
> >>>>>>> >> > type
> >>>>>>> >> > elements
> >>>>>>> >> >  with maxOccour > 1 . I'll keep updating
my progress mean while.
> >>>>>>> >> >
> >>>>>>> >> > Thanks,
> >>>>>>> >> > Shameera.
> >>>>>>> >> >
> >>>>>>> >> > --
> >>>>>>> >> > Shameera Rathnayaka
> >>>>>>> >> > Undergraduate
> >>>>>>> >> > Department of Computer Science and Engineering
> >>>>>>> >> > University of Moratuwa.
> >>>>>>> >> > Sri Lanka.
> >>>>>>> >> >
> >>>>>>> >> > Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>> >> >
> >>>>>>> >>
> >>>>>>> >>
> >>>>>>> >>
> >>>>>>> >> --
> >>>>>>> >> Sagara Gunathunga
> >>>>>>> >>
> >>>>>>> >> Blog      - http://ssagara.blogspot.com
> >>>>>>> >> Web      - http://people.apache.org/~sagara/
> >>>>>>> >> LinkedIn - http://www.linkedin.com/in/ssagara
> >>>>>>> >>
> >>>>>>> >>
> >>>>>>> >> ---------------------------------------------------------------------
> >>>>>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>>>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>>>> >>
> >>>>>>> >
> >>>>>>> >
> >>>>>>> >
> >>>>>>> > --
> >>>>>>> > Amila Suriarachchi
> >>>>>>> > WSO2 Inc.
> >>>>>>> > blog: http://amilachinthaka.blogspot.com/
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Sagara Gunathunga
> >>>>>>>
> >>>>>>> Blog      - http://ssagara.blogspot.com
> >>>>>>> Web      - http://people.apache.org/~sagara/
> >>>>>>> LinkedIn - http://www.linkedin.com/in/ssagara
> >>>>>>>
> >>>>>>> ---------------------------------------------------------------------
> >>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Amila Suriarachchi
> >>>>>> WSO2 Inc.
> >>>>>> blog: http://amilachinthaka.blogspot.com/
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Shameera Rathnayaka
> >>>>> Undergraduate
> >>>>> Department of Computer Science and Engineering
> >>>>> University of Moratuwa.
> >>>>> Sri Lanka.
> >>>>>
> >>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Amila Suriarachchi
> >>>> WSO2 Inc.
> >>>> blog: http://amilachinthaka.blogspot.com/
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> Shameera Rathnayaka
> >>> Undergraduate
> >>> Department of Computer Science and Engineering
> >>> University of Moratuwa.
> >>> Sri Lanka.
> >>>
> >>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>
> >>
> >>
> >>
> >> --
> >> Shameera Rathnayaka
> >> Undergraduate
> >> Department of Computer Science and Engineering
> >> University of Moratuwa.
> >> Sri Lanka.
> >>
> >> Blog : http://shameerarathnayaka.blogspot.com/
> >>
> >
> >
> >
> > --
> > Shameera Rathnayaka
> > Undergraduate
> > Department of Computer Science and Engineering
> > University of Moratuwa.
> > Sri Lanka.
> >
> > Blog : http://shameerarathnayaka.blogspot.com/
> >
> 
> 
> 
> -- 
> Sagara Gunathunga
> 
> Blog      - http://ssagara.blogspot.com
> Web      - http://people.apache.org/~sagara/
> LinkedIn - http://www.linkedin.com/in/ssagara
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
> 
 		 	   		  
Mime
View raw message