commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mario Ivankovits <ma...@ops.co.at>
Subject Re: [VFS] OutOfMemory Exception on large files
Date Wed, 08 Jun 2005 17:45:52 GMT
Torsten Curdt wrote:
>>>We are using HTTP protocol. We have code something like this (in a
>>>simple junit test):-
>>>      
>>Neither "http" nor "url" supports output streams, so I guess your target
>>URL is a "webdav://" resource, isnt it?
>>    
>
>We can even reproduce it with the "file:" protocol!
>  
What? I cant belief.

I tried:

        FileSystemManager fsm = VFS.getManager();
        FileObject foSrc  = fsm.resolveFile("/home/im/vmware/vfstest.tbz2");
        FileObject foDest = 
fsm.resolveFile("/home/im/vmware/vfstest.tmp.tbz2");

        foDest.copyFrom(foSrc, Selectors.SELECT_SELF);

        FileObject foDest2 = 
fsm.resolveFile("/home/im/vmware/vfstest.tmp2.tbz2");


        InputStream is = foSrc.getContent().getInputStream();
        OutputStream os = foDest2.getContent().getOutputStream();

        CopyUtils.copy(is, os);

        os.close();
        is.close();

where vfstest.tbz2 is a 159.726.161 bytes big file. - works like a charme.
As you might see, I tried VFS own copyFrom and IOUtils, but ...
> org.apache.commons.io.IOUtils.copy(is, os);
>  
... which version of IOUtils do you use?
I downloaded 1.0 but instead of finding a  IOUtils.copy I found 
CopyUtils.copy ...

>>Maybe it is possible to use Piped-Streams to pass through to the webdav
>>server, but currently I do not have enough spare time to do this -
>>contributions are welcome :-)
>>    
>
>Can you point me into the right direction ...haven't had
>a closer look into the vfs codebase yet.
>  
Please find:
org.apache.commons.vfs.provider.webdav.WebdavFileObject and its inner 
class WebdavOutputStream

Also the sftp FileProvider uses a byte buffer. Its easy to change this 
to jsch own "getOutputStream()" method but jsch internally also uses a 
pipe and it is dog slow then. Java uses a buffer of 1K for its pipe wich 
seems to be a huge performance bottleneck.


---
Mario

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message