commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Serff <li...@serff.net>
Subject Re: Very Large File Uploads
Date Sat, 26 Aug 2006 19:09:57 GMT
Thanks for the info.  The int for content length is definitely something 
I haven't thought of.  And yes, it was a type. I didn't actually have my 
code with me when I wrote that email, so I was writing it on the fly in 
the email.  :)

So for getName, you recommend I parse the contents for "/" or "\"?  Or 
is there some other way to get the name from the filepart? 

As for the last question, I would like to be able to check on every post 
that the user is allowed to do the upload.  So first thing I would do it 
check their credentials and then process the upload.  If they aren't 
allowed to, I would like to stop them before they even begin to do the 
upload. 

I'd be interested to hear about this streaming API.

Thanks again for the help.
Andrew

Martin Cooper wrote:
> On 8/25/06, Andrew <lists@serff.net> wrote:
>>
>> Hello.  I have seen a few threads about Large file uploads on the
>> mailing list, but most people are saying Large means ~10MB.  I'm trying
>> to upload files larger than 2 GB using FileUpload and HttpClient.  I
>> know some people are going to say "why aren't you using ftp for that?"
>> but I have my reasons, so please try to help me with my problems with
>> FileUpload.  It seems to work ok for files up to about 1GB, but once I'm
>> beyond that, it just hangs forever.  Has anyone had luck doing this with
>> FileUpload?
>
>
> I've done uploads of between 1GB and 2GB before without problems. One 
> thing
> to note, though, is that the Servlet API uses an int for content 
> length, so
> problems can show up once the size is bigger than an int.
>
>  Also, the weird thing is that it seems like the whole file
>> gets there, but parseRequest never returns.  And yes, I've let it sit
>> for hours...
>>
>> This is basically what I have in my doPost method of my servlet:
>> public void doPost(HttpServletRequest request, HttpServletResponse
>> response) throws ServletException, IOException {
>>        String tempStore = System.getProperty("java.iot.tmpdir");
>
>
> Typo in the mail message, or typo in your code? I assume you mean '
> java.io.tmpdir'.
>
>       int maxMemorySize = 10000;
>>        ServletFileUpload sfu = new ServletFileUpload(new
>> DiskFileItemFactory(maxMemorySize, new File(tempStore)));
>>        sfu.setSizeMax(-1);
>>        List fileItems = sfu.parseRequest(request);
>>
>>        Iterator itr = fileItems.iterator();
>>
>>        while(itr.hasNext()) {
>>            FileItem fi = (FileItem)itr.next();
>>            if (!fi.isFormField()) {
>>                File file = new File(tempStore + fi.getName());
>>                fi.write(file);
>>            }
>>        }
>>    }
>>
>> Is there another why I should be doing this?
>
>
> It looks OK, for the most part. However, you shouldn't use getName()
> blindly, as you are, since some browsers will include the path to the
> original file name in that value.
>
> I have one other question too.  Is there a way to parse the request with
>> out accepting the full multipart content?  I'm using the HttpClient to
>> do this upload, and I'm setting a few StringParts in the request so that
>> I can authorize the user to upload.  Doing it the way listed above, I
>> have to accept the entire request (i.e. they upload the whole file)
>> because I get those StringParts.  Should I be putting those parameters
>> somewhere other than the PostMethod I'm sending in the executeMethod of
>> HttpClient?
>
>
> I'm not sure I understand what you want to do. Are you saying you want to
> drop the uploaded file on the floor, but keep the other parameters? If 
> so,
> you might be able to do that with the new streaming API that Jochen 
> added.
> I'm not familiar with it, though, so perhaps he can chime in here.
>
> -- 
> Martin Cooper
>
>
> Any help you all can give would be great!  Thanks!
>> Andrew
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>>
>>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org


Mime
View raw message