axis-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias Wermund" <m.werm...@gg-soft.de>
Subject AW: Async requests killing network
Date Tue, 12 Feb 2008 23:50:51 GMT
That seems to solve the problem.
I have to do some further testing, but a first quick run with
cleanupTransport() after each bunch of requests succeeded.
Also, I could not reproduce the error on Windows Vista systems.

So tomorrow I will investigate this further with the Windows 2000 system.


Thanks,

Matthias

-----Urspr√ľngliche Nachricht-----
Von: Paul Fremantle [mailto:pzfreo@gmail.com] 
Gesendet: Montag, 11. Februar 2008 16:12
An: axis-user@ws.apache.org
Betreff: Re: Async requests killing network

Have you tried using ServiceClient.cleanupTransport() ?

Paul

On Feb 11, 2008 3:01 PM, Matthias Wermund <m.wermund@gg-soft.de> wrote:
> Yes if the requests were made parallel, I'd agree that this is very heavy
> load.
>
> But in my case I do only a few requests parallel, and start the next bunch
> only if they are completed.
> So I assume there is any garbage left from the completed requests?
> That would also explain why the error only appears after some time, and a
> few hundred requests.
>
> ----- Original Message -----
> From: "Paul Fremantle" <pzfreo@gmail.com>
> To: <axis-user@ws.apache.org>
>
> Sent: Monday, February 11, 2008 3:00 PM
> Subject: Re: Async requests killing network
>
>
> > Actually Matthias, 5000 connections is a lot for an untuned OS.
> >
> >  You need to tune your OS to cope with this sort of load. In my
> > personal experience Linux stacks are easier to tune to very high
> > TCP/IP loads. You might want to take a look at the tuning we did here:
> > http://wso2.org/library/2259 (Linux but the principals apply), and
> > this Windows tool as well: http://www.dslreports.com/drtcp
> >
> > Paul
> >
> > On Feb 11, 2008 1:31 PM, Matthias Wermund <m.wermund@gg-soft.de> wrote:
> > > Thanks for your suggestion, I will definately look into this.
> > > Generally spoken, 5000 HTTP connections shouldn't be a problem for
> Windows,
> > > should it?
> > >
> > > Or is this simply a buggy behaviour in the Axis2 HTTP transport?
> > >
> > >
> > > ----- Original Message -----
> > > From: "Paul Fremantle" <pzfreo@gmail.com>
> > > To: <axis-user@ws.apache.org>
> > > Sent: Monday, February 11, 2008 2:06 PM
> > > Subject: Re: Async requests killing network
> > >
> > >
> > > > I should point out you do this by commenting (the normal HTTP
> > > > transport) and uncommenting (the NIO transport) in axis2.xml.
> > > > Some users have also tried the Synapse 1.1.1/1.1.2 NIO HTTP
transport
> > > > which has a number of bugs fixed compared to the Axis2 version
> > > > thereof. You need to grab the JAR file from Synapse and copy the
right
> > > > axis2.xml config from Synapse's axis2.xml.
> > > >
> > > > Paul
> > > >
> > > > On Feb 11, 2008 12:19 PM, Paul Fremantle <pzfreo@gmail.com> wrote:
> > > > > You might want to try this out using the NIO HTTP transport. It
> should
> > > > > scale better for this kind of behaviour.
> > > > >
> > > > > Paul
> > > > >
> > > > >
> > > > > On Feb 11, 2008 11:11 AM, Matthias Wermund <m.wermund@gg-soft.de>
> wrote:
> > > > > >
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I'm having problems starting a big amount of async webservice
> requests
> > > via
> > > > > > Axis2 client in short time.
> > > > > > The requests are all started from the same Thread and should
run
> > > parallel in
> > > > > > only small amount.
> > > > > >
> > > > > > As you can see below, I start about 1000 * 5 = 5000 requests,
but
> only
> > > a max
> > > > > > of 2-5 are called parallel.
> > > > > > For each dataset, a few information-requests are started
parallel,
> but
> > > I
> > > > > > ensure that each CallbackHandler has been finished,
> > > > > > before requesting the informations for the next dataset.
> > > > > >
> > > > > > For the first several hundred requests this works fine, but
after
> some
> > > time,
> > > > > > my network (OS is Windows 2000) goes down,
> > > > > > which means that the complete OS looses the connection to the
> local
> > > network.
> > > > > >
> > > > > > I guess this could be based in opening too much TCP connections
or
> > > something
> > > > > > like this,
> > > > > > because if it affects the whole OS and not just my application,
it
> has
> > > to be
> > > > > > something with the network device or driver.
> > > > > >
> > > > > > So basically my question is:
> > > > > > Is it ensured that the TCP connection is already closed when
> > > > > > CallbackHandler.onError or CallbackHandler.onComplete is called?
> > > > > > Or must I do this myself in any way?
> > > > > > Do you have any other clue what might be the problem here?
> > > > > >
> > > > > > Thank you!
> > > > > >
> > > > > >
> > > > > > ______________________________________________
> > > > > > A quick mock-overview of how I start the threads:
> > > > > >
> > > > > > List<Dataset> datasets;    // approx. 500 - 1000 objects
> > > > > > ...
> > > > > > for (Object data: datasets) {
> > > > > >     List<Callback> callbacks;
> > > > > >     for (Information info: myRequestsForThisData) {    //
approx.
> 2-5
> > > > > > objects
> > > > > >         // Generating the Request Document
> > > > > >         Request request = generateRequest(info);
> > > > > >         // Generating a service Callback instance; the Callback
> class
> > > has an
> > > > > > attribute "finished"
> > > > > >         Callback myCallback = new Callback();
> > > > > >         // remembering the callback
> > > > > >         callbacks.add(myCallback);
> > > > > >         // starting the async request
> > > > > >         stub.startRequest(request,myCallback);
> > > > > >     }
> > > > > >
> > > > > >     // now wait until all Callbacks are finished (finished is
true
> if
> > > > > > onError or onComplete has been called)
> > > > > >     while (true) {
> > > > > >         boolean allCompleted = true;
> > > > > >         for(Callback callback: callbacks)
> > > > > >             if (callback.isFinished()) {
> > > > > >                 allCompleted = false;
> > > > > >                 break;
> > > > > >             }
> > > > > >         if (allCompleted)
> > > > > >             break;
> > > > > >         Thread.sleep(25);
> > > > > >     }
> > > > > > }
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Paul Fremantle
> > > > > Co-Founder and VP of Technical Sales, WSO2
> > > > > OASIS WS-RX TC Co-chair
> > > > >
> > > > > blog: http://pzf.fremantle.org
> > > > > paul@wso2.com
> > > > >
> > > > > "Oxygenating the Web Service Platform", www.wso2.com
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Paul Fremantle
> > > > Co-Founder and VP of Technical Sales, WSO2
> > > > OASIS WS-RX TC Co-chair
> > > >
> > > > blog: http://pzf.fremantle.org
> > > > paul@wso2.com
> > > >
> > > > "Oxygenating the Web Service Platform", www.wso2.com
> > > >
> > > >
---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > > > For additional commands, e-mail: axis-user-help@ws.apache.org
> > > >
> > > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > > For additional commands, e-mail: axis-user-help@ws.apache.org
> > >
> > >
> >
> >
> >
> > --
> > Paul Fremantle
> > Co-Founder and VP of Technical Sales, WSO2
> > OASIS WS-RX TC Co-chair
> >
> > blog: http://pzf.fremantle.org
> > paul@wso2.com
> >
> > "Oxygenating the Web Service Platform", www.wso2.com
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > For additional commands, e-mail: axis-user-help@ws.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>



-- 
Paul Fremantle
Co-Founder and VP of Technical Sales, WSO2
OASIS WS-RX TC Co-chair

blog: http://pzf.fremantle.org
paul@wso2.com

"Oxygenating the Web Service Platform", www.wso2.com

---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org


Mime
View raw message