Mailing-List: contact axis-user-help@ws.apache.org; run by ezmlm
Precedence: bulk
Reply-To: axis-user@ws.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
From: "Matthias Wermund" <m.wermund@gg-soft.de>
To: <axis-user@ws.apache.org>
References: <000e01c86c9e$d4085e90$6400a8c0@jacob.local>
	 <88f5d710802110419scfdd4d6w681a93690449c09c@mail.gmail.com>
	 <88f5d710802110506l1b7968ado5c2e37344ba6991e@mail.gmail.com>
	 <02ac01c86cb2$63bf5080$6400a8c0@jacob.local>
	 <88f5d710802110600w2260b52fv7ebab781fd3f99ea@mail.gmail.com>
	 <000701c86cbf$072afbf0$6400a8c0@jacob.local>
 <88f5d710802110712l36996218j4b186c9b7a38cd07@mail.gmail.com>
In-Reply-To: <88f5d710802110712l36996218j4b186c9b7a38cd07@mail.gmail.com>
Subject: AW: Async requests killing network
Date: Wed, 13 Feb 2008 00:50:51 +0100
Message-ID: <000601c86dd2$19d5f070$4d81d150$@wermund@gg-soft.de>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Thread-Index: AchswI56f0hq03G0R7+ph7rSAtGYQwBETjDg
Content-Language: de

That seems to solve the problem.
I have to do some further testing, but a first quick run with
cleanupTransport() after each bunch of requests succeeded.
Also, I could not reproduce the error on Windows Vista systems.

So tomorrow I will investigate this further with the Windows 2000 =
system.


Thanks,

Matthias

-----Urspr=FCngliche Nachricht-----
Von: Paul Fremantle [mailto:pzfreo@gmail.com]=20
Gesendet: Montag, 11. Februar 2008 16:12
An: axis-user@ws.apache.org
Betreff: Re: Async requests killing network

Have you tried using ServiceClient.cleanupTransport() ?

Paul

On Feb 11, 2008 3:01 PM, Matthias Wermund <m.wermund@gg-soft.de> wrote:
> Yes if the requests were made parallel, I'd agree that this is very =
heavy
> load.
>
> But in my case I do only a few requests parallel, and start the next =
bunch
> only if they are completed.
> So I assume there is any garbage left from the completed requests?
> That would also explain why the error only appears after some time, =
and a
> few hundred requests.
>
> ----- Original Message -----
> From: "Paul Fremantle" <pzfreo@gmail.com>
> To: <axis-user@ws.apache.org>
>
> Sent: Monday, February 11, 2008 3:00 PM
> Subject: Re: Async requests killing network
>
>
> > Actually Matthias, 5000 connections is a lot for an untuned OS.
> >
> >  You need to tune your OS to cope with this sort of load. In my
> > personal experience Linux stacks are easier to tune to very high
> > TCP/IP loads. You might want to take a look at the tuning we did =
here:
> > http://wso2.org/library/2259 (Linux but the principals apply), and
> > this Windows tool as well: http://www.dslreports.com/drtcp
> >
> > Paul
> >
> > On Feb 11, 2008 1:31 PM, Matthias Wermund <m.wermund@gg-soft.de> =
wrote:
> > > Thanks for your suggestion, I will definately look into this.
> > > Generally spoken, 5000 HTTP connections shouldn't be a problem for
> Windows,
> > > should it?
> > >
> > > Or is this simply a buggy behaviour in the Axis2 HTTP transport?
> > >
> > >
> > > ----- Original Message -----
> > > From: "Paul Fremantle" <pzfreo@gmail.com>
> > > To: <axis-user@ws.apache.org>
> > > Sent: Monday, February 11, 2008 2:06 PM
> > > Subject: Re: Async requests killing network
> > >
> > >
> > > > I should point out you do this by commenting (the normal HTTP
> > > > transport) and uncommenting (the NIO transport) in axis2.xml.
> > > > Some users have also tried the Synapse 1.1.1/1.1.2 NIO HTTP
transport
> > > > which has a number of bugs fixed compared to the Axis2 version
> > > > thereof. You need to grab the JAR file from Synapse and copy the
right
> > > > axis2.xml config from Synapse's axis2.xml.
> > > >
> > > > Paul
> > > >
> > > > On Feb 11, 2008 12:19 PM, Paul Fremantle <pzfreo@gmail.com> =
wrote:
> > > > > You might want to try this out using the NIO HTTP transport. =
It
> should
> > > > > scale better for this kind of behaviour.
> > > > >
> > > > > Paul
> > > > >
> > > > >
> > > > > On Feb 11, 2008 11:11 AM, Matthias Wermund =
<m.wermund@gg-soft.de>
> wrote:
> > > > > >
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I'm having problems starting a big amount of async =
webservice
> requests
> > > via
> > > > > > Axis2 client in short time.
> > > > > > The requests are all started from the same Thread and should =
run
> > > parallel in
> > > > > > only small amount.
> > > > > >
> > > > > > As you can see below, I start about 1000 * 5 =3D 5000 =
requests,
but
> only
> > > a max
> > > > > > of 2-5 are called parallel.
> > > > > > For each dataset, a few information-requests are started
parallel,
> but
> > > I
> > > > > > ensure that each CallbackHandler has been finished,
> > > > > > before requesting the informations for the next dataset.
> > > > > >
> > > > > > For the first several hundred requests this works fine, but
after
> some
> > > time,
> > > > > > my network (OS is Windows 2000) goes down,
> > > > > > which means that the complete OS looses the connection to =
the
> local
> > > network.
> > > > > >
> > > > > > I guess this could be based in opening too much TCP =
connections
or
> > > something
> > > > > > like this,
> > > > > > because if it affects the whole OS and not just my =
application,
it
> has
> > > to be
> > > > > > something with the network device or driver.
> > > > > >
> > > > > > So basically my question is:
> > > > > > Is it ensured that the TCP connection is already closed when
> > > > > > CallbackHandler.onError or CallbackHandler.onComplete is =
called?
> > > > > > Or must I do this myself in any way?
> > > > > > Do you have any other clue what might be the problem here?
> > > > > >
> > > > > > Thank you!
> > > > > >
> > > > > >
> > > > > > ______________________________________________
> > > > > > A quick mock-overview of how I start the threads:
> > > > > >
> > > > > > List<Dataset> datasets;    // approx. 500 - 1000 objects
> > > > > > ...
> > > > > > for (Object data: datasets) {
> > > > > >     List<Callback> callbacks;
> > > > > >     for (Information info: myRequestsForThisData) {    //
approx.
> 2-5
> > > > > > objects
> > > > > >         // Generating the Request Document
> > > > > >         Request request =3D generateRequest(info);
> > > > > >         // Generating a service Callback instance; the =
Callback
> class
> > > has an
> > > > > > attribute "finished"
> > > > > >         Callback myCallback =3D new Callback();
> > > > > >         // remembering the callback
> > > > > >         callbacks.add(myCallback);
> > > > > >         // starting the async request
> > > > > >         stub.startRequest(request,myCallback);
> > > > > >     }
> > > > > >
> > > > > >     // now wait until all Callbacks are finished (finished =
is
true
> if
> > > > > > onError or onComplete has been called)
> > > > > >     while (true) {
> > > > > >         boolean allCompleted =3D true;
> > > > > >         for(Callback callback: callbacks)
> > > > > >             if (callback.isFinished()) {
> > > > > >                 allCompleted =3D false;
> > > > > >                 break;
> > > > > >             }
> > > > > >         if (allCompleted)
> > > > > >             break;
> > > > > >         Thread.sleep(25);
> > > > > >     }
> > > > > > }
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Paul Fremantle
> > > > > Co-Founder and VP of Technical Sales, WSO2
> > > > > OASIS WS-RX TC Co-chair
> > > > >
> > > > > blog: http://pzf.fremantle.org
> > > > > paul@wso2.com
> > > > >
> > > > > "Oxygenating the Web Service Platform", www.wso2.com
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Paul Fremantle
> > > > Co-Founder and VP of Technical Sales, WSO2
> > > > OASIS WS-RX TC Co-chair
> > > >
> > > > blog: http://pzf.fremantle.org
> > > > paul@wso2.com
> > > >
> > > > "Oxygenating the Web Service Platform", www.wso2.com
> > > >
> > > >
---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > > > For additional commands, e-mail: axis-user-help@ws.apache.org
> > > >
> > > >
> > >
> > >
> > > =
---------------------------------------------------------------------
> > > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > > For additional commands, e-mail: axis-user-help@ws.apache.org
> > >
> > >
> >
> >
> >
> > --
> > Paul Fremantle
> > Co-Founder and VP of Technical Sales, WSO2
> > OASIS WS-RX TC Co-chair
> >
> > blog: http://pzf.fremantle.org
> > paul@wso2.com
> >
> > "Oxygenating the Web Service Platform", www.wso2.com
> >
> > =
---------------------------------------------------------------------
> > To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> > For additional commands, e-mail: axis-user-help@ws.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
> For additional commands, e-mail: axis-user-help@ws.apache.org
>
>


--=20
Paul Fremantle
Co-Founder and VP of Technical Sales, WSO2
OASIS WS-RX TC Co-chair

blog: http://pzf.fremantle.org
paul@wso2.com

"Oxygenating the Web Service Platform", www.wso2.com

---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: axis-user-unsubscribe@ws.apache.org
For additional commands, e-mail: axis-user-help@ws.apache.org