commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Herber <patrick.her...@gmail.com>
Subject Re: [compress] add a memory efficient stream compress InputStream - e.g. a "DeflaterInputStream"
Date Thu, 03 Sep 2009 09:54:16 GMT
Hello!

Browsing the Internet I found an interesting solution approach to this 
problem using piped input/output streams:

InputStream is = ...;
PipedOutputStream pos = new PipedOutputStream();
PipedInputStream pis = new PipedInputStream(pos);
new DataMover(is, new GZIPOutputStream(pos));
stmt.setBinaryStream(2, pis);
stmt.executeUpdate();

private static class DataMover extends Thread {
    InputStream in = null;
    OutputStream out = null;
    public DataMover(InputStream in, OutputStream out) {
        super("DataMover");
        this.in = in;
        this.out = out;
        start();
    }
    public void run() {
        try {
            byte buf[] = new byte[8192];
            int got = -1;
            while ((got = in.read(buf)) > 0) {
                out.write(buf, 0, got);
            }
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            try {
                in.close();
            } catch (Exception e) {
            }
            try {
                out.close();
            } catch (Exception e) {
            }
        }
    }
}

Regards
Patrick

Patrick Herber wrote:
> Hi!
> Thanks a lot for your reply. Please correct me: in practice do you 
> suggest changing in my piece of code the line
>
> GZIPOutputStream gos = new GZIPOutputStream(new 
> FileOutputStream(tempFile));
>
> with
>
> GzipCompressorOutputStream gos = new GzipCompressorOutputStream(new 
> FileOutputStream(tempFile));isn't it?
>
> In case, I imagined that the solution of the issue COMPRESS-48 was 
> more something that skipped this whole InputStream to OutputStream to 
> InputStream process.
> Something like
>
> InputStream is = ... // original inputStream
> stmt.setBinaryStream(3, new GZIPCompressorInputStream(is), -1);
> // ... "GZIPCompressorInputStream" would be a decorator which compress 
> the underlying inputstream
> stmt.executeUpdate();
>
> Is something not feasible?
>
> Thanks again and best regards,
> Patrick
>
>
> Christian Grobmeier wrote:
>> Hi there,
>>
>> I think this one has been addressed in
>> https://issues.apache.org/jira/browse/COMPRESS-83
>> This change is not released yet. You might want to check the code out
>> from SVN and build it yourself.
>> I have some time left later this year and hopefully this change will
>> come with an 1.1 version soon. But this needs to be discussed on the
>> dev list.
>>
>> Let me know if you need help or if this works out for you
>>
>> Best regards,
>> Christian
>>
>> On Tue, Sep 1, 2009 at 12:33 PM, Patrick Herber<patrick.herber@gmail.com> wrote:
>>   
>>> Hello!
>>>
>>> I've the same issue as described in
>>>
>>> https://issues.apache.org/jira/browse/COMPRESS-48
>>>
>>> My web application recieves an InputStream (an uploaded large file) which I
>>> need to save compressed in the DB.
>>> Currently (since the file could be really big) I save it GZIP-ed into a Temp
>>> File and then I open a new InputStream to that file. However this process
>>> take up more than 60% of the overall method execution:
>>>
>>> tempFile = File.createTempFile(Long.toString(getId()), null);
>>> tempFile.deleteOnExit();
>>> GZIPOutputStream gos = new GZIPOutputStream(new FileOutputStream(tempFile));
>>> IOUtils.copy(is, gos);
>>> is = new FileInputStream(tempFile);
>>> stmt.setBinaryStream(3, is, -1);
>>> stmt.executeUpdate();
>>>
>>> The issue is marked as fixed but I didn't find how/where.
>>> Could you kindly give me a sample how to do it?
>>>
>>> Thanks a lot for your help
>>>
>>> Best regards
>>>
>>> Patrick
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: user-help@commons.apache.org
>>>
>>>
>>>     
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>> For additional commands, e-mail: user-help@commons.apache.org
>>
>>
>>   

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message