commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sebb (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead
Date Sun, 08 Feb 2015 13:13:35 GMT

    [ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311312#comment-14311312
] 

Sebb edited comment on IO-468 at 2/8/15 1:13 PM:
-------------------------------------------------

@[~tn]
The IOUtils class already provides access to methods that accept a user-provided buffer, see
my comment (*) of a few days ago.
These are used by the methods that create the buffer internally.

If there were no downside to using ThreadLocal, I agree that it would make sense to implement
the OP's suggestion.
But as we have both pointed out, there are downsides, and there is an easy work-round.

(*) https://issues.apache.org/jira/browse/IO-468?focusedCommentId=14305390&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14305390


was (Author: sebb@apache.org):
@[~tn]
The IOUtils class already provides access to methods that accept a user-provided buffer, see
my comment (*) of a few days ago.
These are used by the methods that create the buffer internally.

If there were no downside to using ThreadLocal, I agree that it would make sense to implement
the OP's suggestion.
But as we have both pointed out, there are downsides, and there is an easy work-rount.

(*) https://issues.apache.org/jira/browse/IO-468?focusedCommentId=14305390&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14305390

> Avoid allocating memory for method internal buffers, use threadlocal memory instead
> -----------------------------------------------------------------------------------
>
>                 Key: IO-468
>                 URL: https://issues.apache.org/jira/browse/IO-468
>             Project: Commons IO
>          Issue Type: Improvement
>          Components: Utilities
>    Affects Versions: 2.4
>         Environment: all environments
>            Reporter: Bernd Hopp
>            Priority: Minor
>              Labels: newbie, performance
>             Fix For: 2.5
>
>         Attachments: PerfTest.java, monitoring_with_threadlocals.png, monitoring_without_threadlocals.png,
performancetest.ods
>
>   Original Estimate: 12h
>  Remaining Estimate: 12h
>
> In a lot of places, we allocate new buffers dynamically via new byte[]. This is a performance
drawback since many of these allocations could be avoided if we would use threadlocal buffers
that can be reused. For example, consider the following code from IOUtils.java, ln 2177:
> return copyLarge(input, output, inputOffset, length, new byte[DEFAULT_BUFFER_SIZE]);
> This code allocates new memory for every copy-process, that is not used outside of the
method and could easily and safely reused, as long as is is thread-local. So instead of allocating
new memory, a new utility-class could provide a thread-local bytearray like this:
> byte[] buffer = ThreadLocalByteArray.ofSize(DEFAULT_BUFFER_SIZE);
> return copyLarge(input, output, inputOffset, length, buffer);
> I have not measured the performance-benefits yet, but I would expect them to be significant,
especially when the streams itself are not the performance bottleneck. 
> Git PR is at https://github.com/apache/commons-io/pull/6/files



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message