tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Kolinko <knst.koli...@gmail.com>
Subject Re: Problems uploading huge files >2GB to Tomcat app.
Date Thu, 12 Apr 2012 22:13:50 GMT
2012/4/10 Konstantin Kolinko <knst.kolinko@gmail.com>:
> 2012/4/10 Nick Porter <nick.porter@tradar.com>:
>>
>> [NP] Yes. I can fathom no pattern to them. In fact, the only way I can actually see
them is to packet trace the HTTP exchange.
>
> You do not have AccessLogValve configured?!
>
> I did the following to test how the standard Manager application
> handles it.  In 6.0.x and then in current 7.0.x.
>
> 1. Configured AccessLogValve in server.xml as following, to print
> content-length headers of request and response:
>
>  <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
>   prefix="localhost_access_log." suffix=".txt"
>   pattern='%h %l %u %t "%r" %s %b in:[%{content-length}i]
> out:[%{content-length}o]'
>  />
>
> 2. Prepared a two files by concatenating several copies of a zip
> archive and renaming it to *.war.
>
> "test.war" is 127,586,991 bytes
> "test1.war" is 2,314,844,070 bytes
>
> 3. Started Tomcat, logged in into manager and tried to upload the
> files as new web applications.
>
> I am using Firefox 11.0, IE 8.0 both on Windows XP.
>
>
> The results:
> 1). Uploading test.war (127Mb) in Firefox 11
> (...)
> So everything is OK.
>
>
> 2). Uploading test1.war (>2Gb):
>
> Both Tomcat 6 and Tomcat 7 close request immediately, responding with
> status 400.
>
> Looking into access log I see wrong value of content-length header
> from the request.
> The value is different between Firefox and IE, but both are wrong,
> being negative.
>(...)
>

I've tested uploading the >2Gb file (test1.war) into Tomcat Manager
with current Google Chrome.  Result: this browser handles it OK.

I had to change web.xml in manager app in Tomcat 7 to allow upload of
such a file. That is:
[[[
    <multipart-config>
      <max-file-size>-1</max-file-size>
      <max-request-size>-1</max-request-size>
      <file-size-threshold>0</file-size-threshold>
    </multipart-config>
]]]

Result in Tomcat 6 - Failure. The manager displays error message:

FAIL - Deploy Upload Failed, Exception: the request was rejected
because it's size is unknown

In manager.<date>.log file:
[[[
12.04.2012 16:21:33 org.apache.catalina.core.ApplicationContext log
SEVERE: HTMLManager: FAIL - Deploy Upload Failed, Exception: the
request was rejected because it's size is unknown
org.apache.tomcat.util.http.fileupload.FileUploadBase$UnknownSizeException:
the request was rejected because it's size is unknown
	at org.apache.tomcat.util.http.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:267)
	at org.apache.catalina.manager.HTMLManagerServlet.doPost(HTMLManagerServlet.java:190)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
]]]

The code there is based on old version of commons fileupload and does
the following:
(FileUploadBase.java:267)

        int requestSize = req.getContentLength();
        if (requestSize == -1)
        {
            throw new UnknownSizeException(
                "the request was rejected because it's size is unknown");
        }

This code is used in the Manager application only,  so I treat it as
WONTFIX, until there is a real need to deploy 2Gb wars through the
manager. ;)  I am just saying that some old version of Commons
Fileupload did have an issue there.


In Tomcat 7 the code is newer and does handle "-1" properly. There it
more important, because that code is used to implement file upload
support for Servlet 3.0 applications.

In Tomcat 7 manager app  (with the above mentioned configuration
change) the upload succeeded and the archive was unpacked.

Access log:
Tomcat 6:
[[[
127.0.0.1 - tomcat [12/Apr/2012:16:21:33 +0400] "POST
/manager/html/upload?org.apache.catalina.filters.CSRF_NONCE=3F10D7BD7F9D5E31B56A4DB4EF35D96C
HTTP/1.1" 200 14661 in:[2314844272] out:[-]
]]]

Tomcat 7:
[[[
127.0.0.1 - tomcat [12/Apr/2012:17:22:34 +0400] "POST
/manager/html/upload?org.apache.catalina.filters.CSRF_NONCE=E74BBB57E1F12736A9D0357B153BD4A7
HTTP/1.1" 200 17519 in:[2314844272] out:[-]
]]]

For reference:
File size was: 2314844070 bytes
Request size from Chrome was: 2314844272 bytes (= 0x89F9B870 = -1980123024)
Overhead from multipart/form-data was 202 bytes.


(Regarding broken Content-Length in IE: maybe it allocates only 10
chars for their number buffer, so when '-' occupies one character the
whole number gets truncated. Well, a negative value is a garbage
anyway).

Best regards,
Konstantin Kolinko

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message