commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin Cooper" <mart...@apache.org>
Subject Re: Very Large File Uploads
Date Sat, 26 Aug 2006 19:03:27 GMT
On 8/25/06, Andrew <lists@serff.net> wrote:
>
> Hello.  I have seen a few threads about Large file uploads on the
> mailing list, but most people are saying Large means ~10MB.  I'm trying
> to upload files larger than 2 GB using FileUpload and HttpClient.  I
> know some people are going to say "why aren't you using ftp for that?"
> but I have my reasons, so please try to help me with my problems with
> FileUpload.  It seems to work ok for files up to about 1GB, but once I'm
> beyond that, it just hangs forever.  Has anyone had luck doing this with
> FileUpload?


I've done uploads of between 1GB and 2GB before without problems. One thing
to note, though, is that the Servlet API uses an int for content length, so
problems can show up once the size is bigger than an int.

  Also, the weird thing is that it seems like the whole file
> gets there, but parseRequest never returns.  And yes, I've let it sit
> for hours...
>
> This is basically what I have in my doPost method of my servlet:
> public void doPost(HttpServletRequest request, HttpServletResponse
> response) throws ServletException, IOException {
>        String tempStore = System.getProperty("java.iot.tmpdir");


Typo in the mail message, or typo in your code? I assume you mean '
java.io.tmpdir'.

       int maxMemorySize = 10000;
>        ServletFileUpload sfu = new ServletFileUpload(new
> DiskFileItemFactory(maxMemorySize, new File(tempStore)));
>        sfu.setSizeMax(-1);
>        List fileItems = sfu.parseRequest(request);
>
>        Iterator itr = fileItems.iterator();
>
>        while(itr.hasNext()) {
>            FileItem fi = (FileItem)itr.next();
>            if (!fi.isFormField()) {
>                File file = new File(tempStore + fi.getName());
>                fi.write(file);
>            }
>        }
>    }
>
> Is there another why I should be doing this?


It looks OK, for the most part. However, you shouldn't use getName()
blindly, as you are, since some browsers will include the path to the
original file name in that value.

I have one other question too.  Is there a way to parse the request with
> out accepting the full multipart content?  I'm using the HttpClient to
> do this upload, and I'm setting a few StringParts in the request so that
> I can authorize the user to upload.  Doing it the way listed above, I
> have to accept the entire request (i.e. they upload the whole file)
> because I get those StringParts.  Should I be putting those parameters
> somewhere other than the PostMethod I'm sending in the executeMethod of
> HttpClient?


I'm not sure I understand what you want to do. Are you saying you want to
drop the uploaded file on the floor, but keep the other parameters? If so,
you might be able to do that with the new streaming API that Jochen added.
I'm not familiar with it, though, so perhaps he can chime in here.

--
Martin Cooper


Any help you all can give would be great!  Thanks!
> Andrew
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message