Mailing-List: contact dev-help@cxf.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cxf.apache.org
Received-SPF: pass (nike.apache.org: domain of mail2jimma@gmail.com designates
 209.85.212.175 as permitted sender)
MIME-Version: 1.0
Reply-To: mail2jimma@gmail.com
In-Reply-To: <54C7FE71.1030403@gmail.com>
References: <54C7C78B.1010704@redhat.com>
	<341B3282-436A-4C5D-AB27-840096B9461B@apache.org>
	<54C7FE71.1030403@gmail.com>
Date: Wed, 28 Jan 2015 13:45:34 +0800
Message-ID: 
 <CAAvMoHmFYntqbyk300cW-214jcu5J4P4269vpGZbYjf_jOQP6g@mail.gmail.com>
Subject: Re: JAX-WS client performances
From: Jim Ma <mail2jimma@gmail.com>
To: "dev@cxf.apache.org" <dev@cxf.apache.org>
Content-Type: multipart/alternative; boundary=047d7bb049345cbe89050dafe12e

--047d7bb049345cbe89050dafe12e
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

I reported this issue to Oracle several month ago, and it took a long time
that I received the mail from Oracle support team that they need a
producer.
Before I got time to come back to have a look at this again and provided
them code to reproduce this, they already closed as a non-reproducible bug.
I also wrote several emails after that to report this bug still exists in
latest JDK7/8 with some of my analysis, and haven't get their response.

This is another findings I've talked with Oracle: This performance
regression is actually introduced by
https://bugs.openjdk.java.net/browse/JDK-7199862. From changeset
http://hg.openjdk.java.net/jdk7u/jdk7u40/jdk/rev/e6dc1d9bc70b, it tries to
read serverSocket's inputStream to know if the connection is still alive.
But it dramatically slows down the things.

Remove setFixLengthStreamMode() is a temporary workaround, and this will
lead client side consumes more memory for the message doesn't reach default
4k size, especially in some concurrent environment. I agreed with Sergey to
enable/disable this mode with system property and wait JDK team finally get
this fixed.

I created a simple test case to verify if this issue still exists in the
latest jdk if you are interested :
https://github.com/jimma/jdk7u40-httpurlconnection-issue

Cheers,
Jim

On Wed, Jan 28, 2015 at 5:09 AM, Sergey Beryozkin <sberyozkin@gmail.com>
wrote:

> Hi Dan
> On 27/01/15 18:00, Daniel Kulp wrote:
>
>> Chatted with Alessio about this on IRC.   It looks like a bug in the JDK
>> introduced in Java7 update 40.  A bug was logged:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8044726
>>
>> but then closed a non-reproducible.   Alessio will look into seeing if
>> that can be re-opened.
>>
>> Meanwhile we=E2=80=99ll likely remove the call to setFixedLengthStreamin=
gMode.
>> That will result in an extra byte[] copy of the message, but this is onl=
y
>> for smaller messages (hasn=E2=80=99t hit the threshold yet) and thus is =
fairly
>> quick and is certainly MUCH MUCH faster than the problems we=E2=80=99re =
seeing with
>> the call to setFixedLengthStreamingMode().
>>
>>  Should we make it optional (system + contextual property) so that peopl=
e
> can tell the runtime to actually call this method if confirmed it actuall=
y
> improves the performance (may be in newer JDK, etc) ?
>
> Thanks, Sergey
>
>  Dan
>>
>>
>>
>>  On Jan 27, 2015, at 12:14 PM, Alessio Soldano <asoldano@redhat.com>
>>> wrote:
>>>
>>> Hi,
>>> my attention has been recently brought to a scenario in which an Apache
>>> CXF client invokes an endpoint operation in a loop and the number of
>>> invocations performed in a given amount of time (say, 2 minutes) is use=
d as
>>> benchmark for measuring WS stack performances. It's actually a very
>>> simplistic scenario, with a plain JAX-WS single thread client sending a=
nd
>>> receiving small RPC/Lit SOAP messages [1]. The reason why I've been ask=
ed
>>> to have a look is that with default settings the Apache CXF JAX-WS impl
>>> seems to perform *shamefully* bad compared to the Metro (JAX-WS RI)
>>> implementation. I've been blaming the user log configuration, etc but w=
hen
>>> I eventually tried on my own I could actually reproduce the bad results=
.
>>> I've been profiling a bit and found few hot spot area where CXF could
>>> possibly be optimized, but the big issue really seems to be at the
>>> HTTPCounduit / HTTPURLConnection level.
>>> I found that almost all the invocations end up into
>>> sun.net.www.http.HttpClient.New(..) calling available() method [2] as
>>> part of the process for re-using cached connections [3]; that goes to t=
he
>>> wire to try reading and takes a lot of time.
>>> When the RI does the equivalent operation, the available() method is no=
t
>>> called [4], resulting in much better performances.
>>> By looking at the JDK code, it looks to me that the problem boils down
>>> to sun.net.www.protocol.http.HttpURLConnection#streaming() [5]
>>> returning different values, as a consequence of the fixedContentLenght
>>> attribute being set to a value different from -1 when running on CXF on=
ly.
>>> As a matter of fact, that is set when HTTPConduit.WrappedOutputStream#t=
hresholdNotReached()
>>> is called, whenever a message is completely written to the outpustream
>>> buffer before the chunking threshold is reached (at least AFAIU). I've
>>> searched through the JAX-WS RI and could not find any place where
>>> setFixedLengthStreamingMode is called on the connection instead.
>>> So, I've performed two quick and dirty tries: the first time I forced
>>> allowChunking =3D false on the client policy, the second time I comment=
ed out
>>> the code in HTTPConduit.WrappedOutputStream#thresholdNotReached(). In
>>> both cases I managed to get performances comparable to what I can get w=
ith
>>> the JAX-WS RI.
>>> Now, few questions:
>>> - are we really required to call setFixedLengthStreamingMode as we
>>> currently do? what's the drawback of not calling it?
>>> - should we actually do something for getting decent performances by
>>> default in this scenario? (not sure expecting the user to disable chunk=
ing
>>> is that an option...)
>>> As a side note, the relevant part of the JDK HttpClient code changed
>>> between JDK6 and JDK7, so things have not always been as explained abov=
e...
>>>
>>> Cheers
>>> Alessio
>>>
>>>
>>> [1] http://www.fpaste.org/176166/14223765/
>>> [2] http://pasteboard.co/FR5QVrP.png
>>> [3] http://grepcode.com/file/repository.grepcode.com/java/
>>> root/jdk/openjdk/7u40-b43/sun/net/www/http/HttpClient.java#276
>>> [4] http://pasteboard.co/FR8okYM.png
>>> [5] http://grepcode.com/file/repository.grepcode.com/java/
>>> root/jdk/openjdk/7u40-b43/sun/net/www/protocol/http/
>>> HttpURLConnection.java#HttpURLConnection.streaming%28%29
>>>
>>> --
>>> Alessio Soldano
>>> Web Service Lead, JBoss
>>>
>>>
>>
>

--047d7bb049345cbe89050dafe12e--