Mailing-List: contact users-help@tomcat.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Tomcat Users List" <users@tomcat.apache.org>
Received-SPF: pass (athena.apache.org: local policy)
Message-ID: <4F172430.4030809@rosa.nl>
Date: Wed, 18 Jan 2012 20:57:36 +0100
From: Bert Verhees <bert.verhees@rosa.nl>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:8.0) Gecko/20111124 Thunderbird/8.0
MIME-Version: 1.0
To: Tomcat Users List <users@tomcat.apache.org>
Subject: Re: Unable to sendViaPost to url......   ....
 java.net.SocketTimeoutException:
 Read timed out
References: <4F144D12.4000009@rosa.nl> <4F1471A2.4050104@ice-sa.com>
 <4F1484A2.4000102@rosa.nl> <4F1495FB.7090306@pidster.com>
 <4F14AAFA.1030804@ice-sa.com>
In-Reply-To: <4F14AAFA.1030804@ice-sa.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

Sorry, readers of this list for not replying, I am very busy 
investigating the problem, I try replying later.

Bert


On 16-01-12 23:55, Andr� Warnier wrote:
> Hi.
> I'll follow-up with an "out of band" message just for now.
>
> It was not very clear from your initial explanations whether you were 
> sending all of the 1000, 2000, .. 8000 "person's data records" at 
> once, in one HTTP POST, or sending the data in a person-by-person way.
> So my wild guess was based on the symptoms you were providing, which 
> looked very much like a memory problem caused by the processing of 
> increasingly large XML documents using a DOM-like parser.
> And sorry for missing your references to the platform and Tomcat used.
>
> Jumping ahead however, as Pid noted after your memory figures, it 
> would appear that on your system, about 1 GB of swap space on disk 
> /is/ being used, which would be relatively unusual for a Linux system 
> if this application is the only one that is running.
> If indeed this swap space is being used by the Tomcat process, then it 
> would definitely explain why the processing is slow (but the reason 
> why that is so would still need to be found).
> (As an idea : as soon as an application starts to need the swap area, 
> you can easily divide its performance by a factor 10).
>
> It would be interesting to see what the memory usage of the 
> Tomcat-related processes is, while processing these requests.  Your 
> "top" utility would also give an example of this (use CTRL-F, n , <CR> 
> to sort the processes by memory usage).
>
> The general point is : there is no specific reason, per se, why Tomcat 
> would slow down dramatically when processing 1300 requests, as 
> compared to 1000 similar requests.  One would expect the 1300 requests 
> to need approximately 130% of the time needed to process 1000 
> requests, but not the double amount of time.
>
> So if it is not the back-end database, and it is not the Tomcat code 
> per se, that leaves
> - the web application (including its Axis2 components)
> - or some fundamentally different way of handling the data, between
>   - on one side the client + the stand-alone Axis2 server
>   - on the other side the client + the Axis2 Web Service webapp
>   this causing, perhaps, Tomcat to become "saturated" and start trashing
>
>
> I do not really know Axis2, other than what I just read on the 
> Wikipedia page.
> So I can only make some further guesses at this stage :
>
> You mention that when running Axis2 stand-alone, the processing is 
> relatively fast (8000 "persons" in 4 seconds), while when running "the 
> same thing" as a Web service under Tomcat, it is infinitely slower.
> I will assume for now that the background database updates are the 
> same in both cases, and that this is thus not a factor.
>
> You also seem to say that when running Axis2 stand-alone, the 8000 
> "persons" data is only one POST, while through HTTP it would be 8000 
> POSTs (or maybe I misunderstood that also).
>
> In any case, can it be that you are comparing apples with oranges ?
>
> As a loose collection of suggestions :
> - when you send SOAP requests via HTTP, there is an extra layer of 
> wrapping around the SOAP data, both on the client side (wrap as 
> SOAP-HTTP request) and on the server side (unwrap the SOAP-HTTP 
> wrapping).  One would expect some impact of these in any case, if 
> repeated several thousand times in one case, and not in the other.
> - Axis2 seems to allow for configuring entirely different styles of 
> XML parsing (DOM or stream/event-based).  When your client talks to 
> the Axis2 stand-alone server, is it really sending the data the same 
> way, and is the Axis2 server library parsing the data the same way ? 
> Asked another way : is it perhaps so that when you send the data via 
> HTTP to the Web Service, the Web Service code uses a DOM-like parser, 
> while when running in stand-alone mode, the Axis2 server uses a 
> stream-like parsing method ?
> - maybe the Axis2 Web Service webapp library leaks memory ?
> - maybe your webapp using that library leaks memory ?
> - are your client and the server running on the same host ? SOAP is 
> XML. XML processing, particularly in DOM-mode, can use a significant 
> amount of memory (*).  If the client (to create the XML data) and the 
> server (to parse it) are running on the same host, and more or less 
> synchronously, you are doubling the amount of memory required to 
> process your request/response, and that could explain the onset of 
> swapping.
> - when your client sends requests to the Web Servive over HTTP, does 
> it send several requests in parallel ?  How many Tomcat threads are 
> available for processing requests, and how many are effectively 
> processing requests in parallel ?
> - maybe indeed Tomcat is spending a lot of its time doing memory 
> garbage collection, because of one of the above circumstances
>
>
> (*) an estimate is that to represent an XML document in memory using a 
> DOM representation, you easily need 10 times the original amount of 
> data in terms of RAM.
> Stream-style parsing and processing is much faster and much leaner in 
> terms of memory requirements.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org