commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arijit Mukherjee <Arijit.Mukher...@newcastle.ac.uk>
Subject Re: [fielupload] how much big size file can be handled?
Date Wed, 04 Oct 2006 19:57:46 GMT
The machines I'm using are 32-bit (just found that out) - so the API's would
probably limit themselves to a MAX size for counters etc to 2^32-1 which is
2GB. If you search for java.nio issues, you'll get similar things with the
transferTo and TransferFrom API's in the FileChannel.

I was able to upload almost ~10GB on a Mac PowerBook G4, which I guess is
64-bit - with the exact same code - basically the example code in the
documentation.

Arijit


On 4/10/06 15:26, "Andrew" <lists@serff.net> wrote:

> This confuses me.  Why would the VM or the OS matter when streaming a
> file?  Are you trying to stick the entire file into memory?  I'm not
> having any problem now that I am using the 1.2 nightly build.  I
> uploaded a 5GB file yesterday and it worked fine for me.  I'm running on
> RedHat on a 64 bit platform though, but still...I don't get why that
> should matter when all you are doing is reading bytes from one stream
> and writing them to another in small (<2GB) chunks.  Am I missing
> something here?
> 
> Andrew
> Arijit Mukherjee wrote:
>> Leena
>> 
>> Can you please post your code? I wasn't able to find the
>> FileItemIterator in commons-upload-1.1.1 library, but could find it in
>> the nightly builds.
>> 
>> Anyways, I might just have found out reason for the 2GB problem - it
>> probably has something to do with the underlying O/S and the JRE. If the
>> O/S kernel supports 64 bit addressing, then a 2GB limit isn't in the
>> O/S. But you would need the JRE compatible with the 64 bit system to
>> make it work. Otherwise, you have this (2^31-1) limit, which is 2GB.
>> 
>> Regards
>> Arijit
>> 
>>   
>>> -----Original Message-----
>>> From: Leena Kulkarni [mailto:leenakulkarni2003@yahoo.com]
>>> Sent: 04 October 2006 08:29
>>> To: Jakarta Commons Users List
>>> Subject: Re: [fielupload] how much big size file can be handled?
>>> 
>>> Actually we are facing problems for much less size files than
>>> you all mentioned here. Its taking too long for 35MB files only..
>>> 
>>> Arijit, we are using same code like yours but we get items as
>>> empty at around 60MB file.
>>> 
>>> 1. Is there anything wrong that we are doing?
>>> 2. We tried the streaming API, FileItemIterator is not
>>> available in the jar.
>>> 
>>> Any help?
>>> 
>>> Regards,
>>> Leena
>>> --- Martin Cooper <martinc@apache.org> wrote:
>>> 
>>>     
>>>> On 10/3/06, Jochen Wiedmann
>>>> <jochen.wiedmann@gmail.com> wrote:
>>>>       
>>>>> On 10/3/06, Martin Cooper <martinc@apache.org>
>>>>>         
>>>> wrote:
>>>>       
>>>>>>> Also, it looks like UnknownSizeException has
>>>>>>>            
>>>> been deprecated because
>>>>       
>>>>> it
>>>>>         
>>>>>>> doesn't exist in the latest release.
>>>>>>>            
>>>>>> It is not deprecated and should not have been
>>>>>>           
>>>> removed. It seems that it
>>>>       
>>>>> was
>>>>>         
>>>>>> removed by Jochen (cc'd) when he merged in his
>>>>>>           
>>>> streaming changes. He
>>>>       
>>>>> needs
>>>>>         
>>>>>> to put it back, though, since it is part of the
>>>>>>           
>>>> 1.x public API.
>>>>       
>>>>>> Note that UnknownSizeException is still part of
>>>>>>           
>>>> the latest official
>>>>       
>>>>> release
>>>>>         
>>>>>> (1.1.1). It it not in the nightly builds,
>>>>>>           
>>>> however.
>>>>       
>>>>> Yes, I removed it. But what good does the
>>>>>         
>>>> exception, if there's no
>>>>       
>>>>> code that throws it?
>>>>>         
>>>> The fact is that throwing that exception when the request is
>>>>       
>>> too large 
>>>     
>>>> is part of the FileUpload 1.x API contract. With your changes, that
>>>> contract is now broken. If the next release of FileUpload is
>>>>       
>>> going to 
>>>     
>>>> be a 1.x release, the contract must be honoured. Otherwise, the
>>>> current code in trunk must be considered part of a 2.0 release.
>>>> 
>>>> --
>>>> Martin Cooper
>>>> 
>>>> 
>>>> Jochen
>>>>       
>>>>> --
>>>>> My wife Mary and I have been married for
>>>>>         
>>>> forty-seven years and not
>>>>       
>>>>> once have we had an argument serious enough to
>>>>>         
>>>> consider divorce;
>>>>       
>>>>> murder, yes, but divorce, never.
>>>>> (Jack Benny)
>>>>> 
>>>>>         
>>> __________________________________________________
>>> Do You Yahoo!?
>>> Tired of spam?  Yahoo! Mail has the best spam protection
>>> around http://mail.yahoo.com
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>>> 
>>> 
>>>     
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>> 
>>   
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org


Mime
View raw message