cxf-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted <r6squee...@gmail.com>
Subject Re: gzip compression issues with threshold and client compression
Date Thu, 14 Mar 2013 01:04:16 GMT
I hadn't thought about that, so I decided to check :)

Simple test case : 1 web service call - which in http normally results
in 4 tcp transmissions (2 incoming and 2 out going, the 2 because one
of them is the wsdl request, doesn't participate in GZ but for
simplicity of calculation it's included).

When I turn on SSL with out GZ this turns into 18 TCP transmissions, I
can see at least  6 being part of the SSL handshake/initialisation.
The sum of the 12 application transmission sizes = 21,524 bytes.

When I turn on SSL with GZ this turns only into 16 TCP transmissions,
still with 6 being from the SSL handshake. Why this is... I don't
know.  The sum of the 10 application transmission sizes = 4,230 bytes


I can consistently reproduce the 18 v.s. 16 transmission difference,
and I can consistently reproduce results approximating 21k v.s. 4k.

So yes, it appears to  have a benefit even with SSL.

On 3/14/13, Glen Mazza <gmazza@talend.com> wrote:
> I'm unsure whether SSL does any compression of its own, I would guess it
> would in order to minimize the amount of time needed to encrypt.  You
> might wish to check if there's any real performance difference between
> SSL and SSL + gzip.
>
> Glen
>
> On 03/12/2013 09:56 PM, Ted wrote:
>> thanks all, it appears in http using wireshark (finally got it to show
>> me what I wanted) it is doing what is expected. Https... well too hard
>> for me to check so I'll just take it on faith that it works since it's
>> just a tomcat connector outside of what cxf/my application returns.
>>
>>
>>
>>
>> On 3/13/13, Ted <r6squeegee@gmail.com> wrote:
>>> thanks, I'd looked at the blog before, I don't believe it has anything
>>> I haven't already implemented.
>>>
>>> In terms of wireshark, I had also tried that before but none of the
>>> contents were comprehensible to me so I was not really able to tell
>>> what was or wasn't gzipped. I even moved it from https to http to try
>>> to get the contents in a relatively "plain text" format so I can see
>>> when it was not compressed and when it was, but I could never get the
>>> plain text version so I was not able to determine a change... it's the
>>> first time I'd used wireshark though, that's why I tried to look at
>>> the header instead.
>>>
>>> The header shows up on larger packets so I suspect at least for the
>>> threshold problem on out bound messages, it should be an indication of
>>> when it is and is not gzipped.
>>>
>>> I'll spend some more time with wireshark and see if I can't figure out
>>> how to use it a little better though. thanks.
>>>
>>> On 3/13/13, Daniel Kulp <dkulp@apache.org> wrote:
>>>> Can you use wireshark or similar to get the raw transfer bytes?   I
>>>> believe
>>>> the GZIPInInterceptor would run before the LoggingInInterceptor.  Thus,
>>>> but
>>>> the time it gets to the logging interceptor, the payload has been
>>>> completely
>>>> un-gzipped and the Content-Encoding header stripped off.
>>>>
>>>> Dan
>>>>
>>>>
>>>>
>>>>
>>>> On Mar 11, 2013, at 8:56 PM, Ted <r6squeegee@gmail.com> wrote:
>>>>
>>>>> Thanks, I'm not trying to force the first one to be GZ, but the
>>>>> problem is the second one doesn't appear to be GZ either.
>>>>>
>>>>> Just to test it, I hard coded 2 identical calls in succession :
>>>>>
>>>>> 	System.err.println(messageWs.getUnreadActiveMessageCount(loggedInInfo.getLoggedInPersonId()));
>>>>> 	System.err.println(messageWs.getUnreadActiveMessageCount(loggedInInfo.getLoggedInPersonId()));
>>>>>
>>>>> The out put from the logs appear to be non gz-ed :
>>>>> 	2013-03-12 11:31:14,818 INFO  [MessageWs:234] Inbound Message
>>>>> 	----------------------------
>>>>> 	ID: 38
>>>>> 	Address: https://127.0.0.1:8091/myoscar_server/ws/MessageService
>>>>> 	Encoding: UTF-8
>>>>> 	Http-Method: POST
>>>>> 	Content-Type: text/xml; charset=UTF-8
>>>>> 	Headers: {Accept=[*/*], accept-encoding=[gzip;q=1.0, identity; q=0.5,
>>>>> *;q=0], cache-control=[no-cache], connection=[keep-alive],
>>>>> Content-Length=[785], content-type=[text/xml; charset=UTF-8],
>>>>> host=[127.0.0.1:8091], pragma=[no-cache], SOAPAction=[""],
>>>>> user-agent=[Apache CXF 2.7.3]}
>>>>> 	Payload: <soap:Envelope ... </soap:Envelope>
>>>>> 	--------------------------------------
>>>>> 	2013-03-12 11:31:14,834 INFO  [MessageWs:234] Outbound Message
>>>>> 	---------------------------
>>>>> 	ID: 38
>>>>> 	Encoding: UTF-8
>>>>> 	Content-Type: text/xml
>>>>> 	Headers:
>>>>> 	Payload: <soap:Envelope
>>>>> xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Body><ns2:getUnreadActiveMessageCountResponse
>>>>> xmlns:ns2="http://ws.myoscar_server.oscarehr.org/"><return>31</return></ns2:getUnreadActiveMessageCountResponse></soap:Body></soap:Envelope>
>>>>> 	--------------------------------------
>>>>> 	2013-03-12 11:31:14,841 INFO  [MessageWs:234] Inbound Message
>>>>> 	----------------------------
>>>>> 	ID: 39
>>>>> 	Address: https://127.0.0.1:8091/myoscar_server/ws/MessageService
>>>>> 	Encoding: UTF-8
>>>>> 	Http-Method: POST
>>>>> 	Content-Type: text/xml; charset=UTF-8
>>>>> 	Headers: {Accept=[*/*], accept-encoding=[gzip;q=1.0, identity; q=0.5,
>>>>> *;q=0], cache-control=[no-cache], connection=[keep-alive],
>>>>> Content-Length=[785], content-type=[text/xml; charset=UTF-8],
>>>>> host=[127.0.0.1:8091], pragma=[no-cache], SOAPAction=[""],
>>>>> user-agent=[Apache CXF 2.7.3]}
>>>>> 	Payload: <soap:Envelope ... </soap:Envelope>
>>>>> 	--------------------------------------
>>>>> 	2013-03-12 11:31:14,855 INFO  [MessageWs:234] Outbound Message
>>>>> 	---------------------------
>>>>> 	ID: 39
>>>>> 	Encoding: UTF-8
>>>>> 	Content-Type: text/xml
>>>>> 	Headers:
>>>>> 	Payload: <soap:Envelope
>>>>> xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Body><ns2:getUnreadActiveMessageCountResponse
>>>>> xmlns:ns2="http://ws.myoscar_server.oscarehr.org/"><return>31</return></ns2:getUnreadActiveMessageCountResponse></soap:Body></soap:Envelope>
>>>>> 	--------------------------------------
>>>>>
>>>>>
>>>>> I'm assuming I should have seen a "Content-Encoding=[gzip]" in the
>>>>> header somewhere if it were gz-ed?
>>>>>
>>>>> Just to see if it's related to the 1k theshold problem, I did a 4k
>>>>> message test too, still same header results, no gz.
>>>>>
>>>>> I also tried your suggestion just to test :
>>>>>
>>>>> 	GZIPOutInterceptor x = new GZIPOutInterceptor(100);
>>>>> 	x.setForce(true);
>>>>> 	System.err.println("----- JUST SET FORCE TRUE");
>>>>> 	cxfClient.getOutInterceptors().add(x);
>>>>>
>>>>> still the same results. Is it suppose to show
>>>>> "Content-Encoding=[gzip]" on the inbound requests? or is there another
>>>>> way to check for gz?
>>>>>
>>>>> Also the outbound requests are not taking the threshold into account,
>>>>> it seems to be 1k no matter what I set it to. Notice the responses
>>>>> from above are also missing the gzip entry in the header, I do get
>>>>> those on larger outbound objects though so I know the setting is at
>>>>> least taking effect sometimes.
>>>>>
>>>>> On 3/12/13, Daniel Kulp <dkulp@apache.org> wrote:
>>>>>> By default, the GZIP out stuff on the client uses a "negotiated"
>>>>>> method
>>>>>> where the first request is not-gzipped, but with the Accept-Encoding
>>>>>> header
>>>>>> since it does not know if the server knows how to handle a GZIP post.
>>>>>> If
>>>>>> the server responds with a GZIP response, subsequent requests from
>>>>>> that
>>>>>> client will be gzipped.     Try making multiple calls with the same
>>>>>> client
>>>>>> object and see if the headers and such change.
>>>>>>
>>>>>> You can force the first request to be gzipped by doing:
>>>>>>
>>>>>> GZIPOutInterceptor gz = new GZIPOutInterceptor(128)
>>>>>> gz.setForce(true);
>>>>>> client.getOutInterceptors().add(gz);
>>>>>>
>>>>>> Hope that helps!
>>>>>>
>>>>>> Dan
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mar 11, 2013, at 12:22 AM, Ted <r6squeegee@gmail.com> wrote:
>>>>>>
>>>>>>> Hi I just tried gzip compression and it appears to be working
in
>>>>>>> some
>>>>>>> cases but not in others.
>>>>>>> I'm on openjdk 1.7.0.3 and cxf 2.7.3 and tomcat 7.0.27.
>>>>>>>
>>>>>>> On the server side I've set
>>>>>>> 	@GZIP(threshold=128)
>>>>>>> on the web services
>>>>>>>
>>>>>>> On the client I've set
>>>>>>> 	cxfClient.getInInterceptors().add(new GZIPInInterceptor());
>>>>>>> 	cxfClient.getOutInterceptors().add(new GZIPOutInterceptor(128));
>>>>>>>
>>>>>>> I'm logging what's going on with the server with :
>>>>>>> 	<bean id="compressGZIPFeature"
>>>>>>> class="org.apache.cxf.transport.common.gzip.GZIPFeature"/>
>>>>>>> 	<bean id="loggingFeature"
>>>>>>> class="org.apache.cxf.feature.LoggingFeature"/>
>>>>>>>
>>>>>>> 	<cxf:bus>
>>>>>>> 		<cxf:features>
>>>>>>> 			<ref bean="compressGZIPFeature"/>
>>>>>>> 			<ref bean="loggingFeature"/>
>>>>>>> 		</cxf:features>
>>>>>>> 	</cxf:bus>
>>>>>>>
>>>>>>> What I can see is the client is sending
>>>>>>> 	Headers: {Accept-Encoding=[gzip;q=1.0, identity; q=0.5, *;q=0]}
>>>>>>>
>>>>>>> I can see if I have a large response (over 1k) I'm getting
>>>>>>> 	Headers: {Content-Encoding=[gzip], Vary=[Accept-Encoding]}
>>>>>>>
>>>>>>> The 2 problems I'm having is
>>>>>>>
>>>>>>> 1) the 128 seems to have no effect, I even tried '0' as per the
>>>>>>> documentation for always on, and it does seem to work. Small
data is
>>>>>>> just normal / no mention of gzip encoding like the large data
>>>>>>>
>>>>>>> 2) The requests being sent by the client do not appear to be
gziped,
>>>>>>> but honestly I'm not sure how to check or what header to look
for.
>>>>>>> Have I missed something in the configuration?
>>>>>>>
>>>>>>> thanks
>>>>>>> --
>>>>>>> Ted.
>>>>>> --
>>>>>> Daniel Kulp
>>>>>> dkulp@apache.org - http://dankulp.com/blog
>>>>>> Talend Community Coder - http://coders.talend.com
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Ted.
>>>> --
>>>> Daniel Kulp
>>>> dkulp@apache.org - http://dankulp.com/blog
>>>> Talend Community Coder - http://coders.talend.com
>>>>
>>>>
>>>
>>> --
>>> Ted.
>>>
>>
>
>


-- 
Ted.

Mime
View raw message