commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IO-515) Allow Specifying Initial Buffer Size of DeferredFileOutputStream
Date Tue, 27 Sep 2016 20:44:20 GMT

    [ https://issues.apache.org/jira/browse/IO-515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15527366#comment-15527366
] 

ASF GitHub Bot commented on IO-515:
-----------------------------------

GitHub user brettlounsbury opened a pull request:

    https://github.com/apache/commons-io/pull/22

    [IO-515] Add constructors that allow the initial size of the in memory buffer

    Add constructors that allow the initial size of the in memory buffer to be set to a value
other than the default value in ByteArrayOutputStream.  The current default is 32B which can
have large performance impact if the threshold is large enough due to multiple array doubling
iterations.  As an example, to buffer 1MB the array would double 15 times and would create
approximately 1MB of garbage arrays in addition to the 1MB of buffer at the end.
    
    https://issues.apache.org/jira/browse/IO-515

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/brettlounsbury/commons-io master

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/commons-io/pull/22.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #22
    
----
commit 112d6c02a4c0d3fe9caa658505f31da3908b1467
Author: Brett Lounsbury <brettl@amazon.com>
Date:   2016-09-27T19:58:21Z

    Add constructors that allow the initial size of the in memory buffer to be set to a value
other than the default value in ByteArrayOutputStream.  The current default is 32B which can
have large performance impact if the threshold is large enough due to multiple array doubling
iterations.  As an example, to buffer 1MB the array would double 15 times and would create
approximately 1MB of garbage arrays in addition to the 1MB of buffer at the end.

----


> Allow Specifying Initial Buffer Size of DeferredFileOutputStream
> ----------------------------------------------------------------
>
>                 Key: IO-515
>                 URL: https://issues.apache.org/jira/browse/IO-515
>             Project: Commons IO
>          Issue Type: Improvement
>          Components: Streams/Writers
>            Reporter: Brett Lounsbury
>
> Right now the DeferredFileOutputStream uses a ByteArrayOputStream for its in memory buffer.
 It simply does new ByteArrayOutputStream() which initializes the ByteArrayOutputStream's
internal buffer to 32B.  In the case of large threshold values for in-memory buffering this
can become very inefficient (Requiring 15 array doubling operations to grow to 1MB and producing
an extra 1MB of garbage along the way).
> By allowing the user to specify the initial buffer size we allow them to tune how many
times the array will need to double to something that meets their application's needs.
> In our testing we had an application that used a DeferredFileOutputStream to capture
incoming data.  Once that data was fully captured it did some fairly CPU intensive processing
of the data.  The ByteArrayOutputStream used by DeferredFileOutputStream was using 60% of
our host's CPU simply on array copying.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message