commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin Cooper" <mart...@apache.org>
Subject Re: Very Large File Uploads
Date Sat, 26 Aug 2006 19:19:27 GMT
On 8/26/06, Andrew Serff <lists@serff.net> wrote:
>
> Thanks for the info.  The int for content length is definitely something
> I haven't thought of.  And yes, it was a type. I didn't actually have my
> code with me when I wrote that email, so I was writing it on the fly in
> the email.  :)
>
> So for getName, you recommend I parse the contents for "/" or "\"?  Or
> is there some other way to get the name from the filepart?


There's a FAQ entry that shows a few lines of code:

http://jakarta.apache.org/commons/fileupload/faq.html

As for the last question, I would like to be able to check on every post
> that the user is allowed to do the upload.  So first thing I would do it
> check their credentials and then process the upload.  If they aren't
> allowed to, I would like to stop them before they even begin to do the
> upload.


Ah, OK. Other than the possibility of the streaming API, you could "cheat"
and use the URL for additional parameters. The FileUpload API deals only
with the actual form content. If you add regular request parameters to the
URL you post to, you can access those using the plain old servlet API (i.e.
request.getParameter() and friends). If you don't want to allow the upload,
you can then just skip calling FileUpload at all.

--
Martin Cooper


I'd be interested to hear about this streaming API.
>
> Thanks again for the help.
> Andrew
>
> Martin Cooper wrote:
> > On 8/25/06, Andrew <lists@serff.net> wrote:
> >>
> >> Hello.  I have seen a few threads about Large file uploads on the
> >> mailing list, but most people are saying Large means ~10MB.  I'm trying
> >> to upload files larger than 2 GB using FileUpload and HttpClient.  I
> >> know some people are going to say "why aren't you using ftp for that?"
> >> but I have my reasons, so please try to help me with my problems with
> >> FileUpload.  It seems to work ok for files up to about 1GB, but once
> I'm
> >> beyond that, it just hangs forever.  Has anyone had luck doing this
> with
> >> FileUpload?
> >
> >
> > I've done uploads of between 1GB and 2GB before without problems. One
> > thing
> > to note, though, is that the Servlet API uses an int for content
> > length, so
> > problems can show up once the size is bigger than an int.
> >
> >  Also, the weird thing is that it seems like the whole file
> >> gets there, but parseRequest never returns.  And yes, I've let it sit
> >> for hours...
> >>
> >> This is basically what I have in my doPost method of my servlet:
> >> public void doPost(HttpServletRequest request, HttpServletResponse
> >> response) throws ServletException, IOException {
> >>        String tempStore = System.getProperty("java.iot.tmpdir");
> >
> >
> > Typo in the mail message, or typo in your code? I assume you mean '
> > java.io.tmpdir'.
> >
> >       int maxMemorySize = 10000;
> >>        ServletFileUpload sfu = new ServletFileUpload(new
> >> DiskFileItemFactory(maxMemorySize, new File(tempStore)));
> >>        sfu.setSizeMax(-1);
> >>        List fileItems = sfu.parseRequest(request);
> >>
> >>        Iterator itr = fileItems.iterator();
> >>
> >>        while(itr.hasNext()) {
> >>            FileItem fi = (FileItem)itr.next();
> >>            if (!fi.isFormField()) {
> >>                File file = new File(tempStore + fi.getName());
> >>                fi.write(file);
> >>            }
> >>        }
> >>    }
> >>
> >> Is there another why I should be doing this?
> >
> >
> > It looks OK, for the most part. However, you shouldn't use getName()
> > blindly, as you are, since some browsers will include the path to the
> > original file name in that value.
> >
> > I have one other question too.  Is there a way to parse the request with
> >> out accepting the full multipart content?  I'm using the HttpClient to
> >> do this upload, and I'm setting a few StringParts in the request so
> that
> >> I can authorize the user to upload.  Doing it the way listed above, I
> >> have to accept the entire request (i.e. they upload the whole file)
> >> because I get those StringParts.  Should I be putting those parameters
> >> somewhere other than the PostMethod I'm sending in the executeMethod of
> >> HttpClient?
> >
> >
> > I'm not sure I understand what you want to do. Are you saying you want
> to
> > drop the uploaded file on the floor, but keep the other parameters? If
> > so,
> > you might be able to do that with the new streaming API that Jochen
> > added.
> > I'm not familiar with it, though, so perhaps he can chime in here.
> >
> > --
> > Martin Cooper
> >
> >
> > Any help you all can give would be great!  Thanks!
> >> Andrew
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> >> For additional commands, e-mail: commons-user-help@jakarta.apache.org
> >>
> >>
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message