hc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pani, Gourav" <gourav.p...@brightpoint.com>
Subject RE: Problem encountered posting large files
Date Tue, 11 Mar 2003 17:30:38 GMT
i guess the attached zip file didn't go through.  how about these ones?



-----Original Message-----
From: Pani, Gourav [mailto:gourav.pani@brightpoint.com]
Sent: Tuesday, March 11, 2003 12:20 PM
To: 'Commons HttpClient Project'
Subject: RE: Problem encountered posting large files


Oleg,

We are able to post internally to an echo servlet (running on Resin) without
any problems, so the problem is likely with the configuration of our
partner's application server.  Unfortunately we can't get the required
resources to look into the problem on their side since we already have a
solution working with them.

At this point we are going to keep sending the data using the same code we
are currently using (the socket code listed below) but we thought we'd
include the rest of our research in case it might be able to shed some light
on any possible issues.

Attached are the trace files (IP addresses masked) from sending the same
exact file to our partner (external.output) and to our echo servlet
internally (internal.output).  Clearly it took MUCH longer to send the same
file to our partner than it did to send it internally. But what is really
confusing is that using the socket code we have listed below it takes less
than 30 seconds to send the same file to our partner so that rules out any
network latency.


Thanks,

Gourav



-----Original Message-----
From: Kalnichevski, Oleg [mailto:oleg.kalnichevski@bearingpoint.com]
Sent: Tuesday, March 11, 2003 9:49 AM
To: Commons HttpClient Project
Subject: RE: Problem encountered posting large files


Gourav,

I just concatenated RFC 2616 six times and posted the resulting file (2.5
MB) to an 'echo' servlet running on Tomcat 4.1.18 using PostXML demo
application. The response came back almost instantaneously and overall
process including printing out the response body to the console under
Eclipse took no more than a few seconds. Unless the word of RFC 2616 has
magical bearing upon HTTP protocol related bugs, like that of the Holy Bible
on (non-unix) demons, the problem does appear to be specific to your local
environment. 

Try running your application with logging on, and see if you find any clues
there. 

http://jakarta.apache.org/commons/httpclient/logging.html

You are welcome to post the resultant wire log to this mailing lost for
scrutiny.

Folks, any other ideas?

Oleg

-----Original Message-----
From: Pani, Gourav [mailto:gourav.pani@brightpoint.com]
Sent: Dienstag, 11. März 2003 15:08
To: 'Commons HttpClient Project'
Subject: RE: Problem encountered posting large files


Oleg,

I can reliably reproduce the problem any time we try posting a large xml
file (> 1 MB).  I'm not sure that I could produce a JUnit test case because
the code executes fine, it simply takes an exponentially long time than it
does using the current code that simply opens a socket and sends the data
(that code is included below).

      // Get URL Connection and Socket
      url = new URL(this.url);
      socket = new Socket(url.getHost(), url.getPort());
      socket.setSoTimeout(BestProperties.getIntProperty("vmu.url.timeout"));
      
      // building request in StringBuffer
      sbRequest.setLength(0);
      sbRequest.append("POST" + " " + url.getFile() + " HTTP/1.0\r");
      sbRequest.append("Content-Type: text/xml\r");
      sbRequest.append("User-Agent: Java 1.4.0\n");
      sbRequest.append("Host: " + url.getHost() + "\n");
      sbRequest.append("Accept: text/html,text/xml,text/plain, */*\n");
      sbRequest.append("Cache-Control: no-cache\n");
      sbRequest.append("Connection: Close\n");
      sbRequest.append("Content-Length: " + stream.length() + "\n");
      sbRequest.append("\n" + stream + "\n");

      // sending the request
      request = new PrintStream(new
BufferedOutputStream(socket.getOutputStream()));
      request.println(sbRequest.toString());
      request.flush();
      // getting response from client
      response = new BufferedReader(new
InputStreamReader(socket.getInputStream()));



- what platform are you using on the client-side and on the server-side?
  We are using Linux on the client and Solaris on the server

- are you using chunk-encoding or not? 
- if not, are you explicitly specifying the content length or not?  
- are you sure that HttpClient does try to buffer the request in order to be
able to determine its size? 
  We are using the PostXML example from the CVS tree.  We made one
modification and that was to include a debug statement so that we knew that
it was setting the ContentLenght to:  2022007

- have you tried posting large files to Tomcat servlet engine running on the
same platform? I have not tried it myself, but I believe quite a few
people have been successfully using HttpClient to post massive files
  Unfortunately this is not an option for us at this time.

-----Original Message-----
From: Oleg Kalnichevski [mailto:o.kalnichevski@dplanet.ch]
Sent: Monday, March 10, 2003 5:39 PM
To: Commons HttpClient Project
Subject: RE: Problem encountered posting large files


Gourav,

We will have difficulties addressing this problem unless we know how it
can be reliably reproduced. So, we will need your help on that. A JUnit
test case reproducing the problem would be ideals

More questions:
- what platform are you using on the client-side and on the server-side?
- are you using chunk-encoding or not? 
- if not, are you explicitly specifying the content length or not? 
- are you sure that HttpClient does try to buffer the request in order
to be able to determine its size? 
- have you tried posting large files to Tomcat servlet engine running on
the same platform? I have not tried it myself, but I believe quite a few
people have been successfully using HttpClient to post massive files

Cheers

Oleg 


On Mon, 2003-03-10 at 23:06, Pani, Gourav wrote:
> Oleg,
> 
> -  The HttpClient takes a lot longer to post large files as opposed to
using
> a raw socket.
> -  The header on the HTTP Server is as follows
> 	HTTP/1.0 200 OK
> 	Date: Mon, 10 Mar 2003 22:02:57 GMT
> 	Server: WebLogic WebLogic Temporary Patch for CR072964 04/03/2002
> 10:26:28
> 	Content-Length: 0
> 	Content-Type: text/xml
> 	Connection: Close
> -  We do not get any exceptions because it keeps trying to post and then
the
> thread hangs.  Finally, I just have to kill the thread.  
> 
> 
> Thanks,
> Gourav
> 
> 
> -----Original Message-----
> From: Oleg Kalnichevski [mailto:o.kalnichevski@dplanet.ch]
> Sent: Monday, March 10, 2003 5:00 PM
> To: Commons HttpClient Project
> Subject: Re: Problem encountered posting large files
> 
> 
> Gourav,
> 
> Help us understand the problem better:
> - Are you saying that it takes longer to post a large file using
> HttpClient compared to using a raw socket?
> - What kind of HTTP server are you posting your request to?
> - What kind of exception are you getting? Can you post the the call
> stack of that exception?
> 
> Oleg
> 
> On Mon, 2003-03-10 at 22:36, Pani, Gourav wrote:
> > I am using HttpClient release 2.0 Alpha 3 using J2SDK1.4.1_01 .  When I
> try
> > posting large files greater than 2MB using the PostXML.java example, I
see
> > the packets going across and get responses back but it takes extremely
> long
> > sending the data and in the process, the connection times out.  
> > 
> > I managed to successfully post and receive a response using the Socket
> > object in a different test program.
> > 
> > Any help is greatly appreciated.
> > 
> > Thanks in Advance.
> > 
> > Gourav
> > 
> > 
> > 
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> commons-httpclient-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> commons-httpclient-dev-help@jakarta.apache.org
> > 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> commons-httpclient-dev-help@jakarta.apache.org
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
commons-httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
commons-httpclient-dev-help@jakarta.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail:
commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail:
commons-httpclient-dev-help@jakarta.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail:
commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail:
commons-httpclient-dev-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail:
commons-httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail:
commons-httpclient-dev-help@jakarta.apache.org




Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message