commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bernd Hopp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IO-468) Avoid allocating memory for method internal buffers, use threadlocal memory instead
Date Thu, 05 Feb 2015 09:52:35 GMT

    [ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306930#comment-14306930
] 

Bernd Hopp commented on IO-468:
-------------------------------

I referred to the current PR at https://github.com/apache/commons-io/pull/6/files

Freeing up threadlocal resources after a thread ended is up to the java runtime, so it may
or may not be released immediately after a thread ends. Anyway, I don't see any scenario where
this could be a problem. The only way to get a lesser throughput with the new code would be
to create a new thread outside a threadpool and let this thread invoke e.g. copy(InputStream,
OutputStream) a single time. But even in this case, I assume the performance cost to be neglectable
compared to the benefits it gains in more common scenarios.

The fact that IOUtils already allows users to supply the buffer themself is to me not an argument
against a patch that gives significant improvements with no api-breakage. Why would you expect
users to implement performance-optimizations themself if the framework could easily incorporate
them? 

Regards
Bernd

> Avoid allocating memory for method internal buffers, use threadlocal memory instead
> -----------------------------------------------------------------------------------
>
>                 Key: IO-468
>                 URL: https://issues.apache.org/jira/browse/IO-468
>             Project: Commons IO
>          Issue Type: Improvement
>          Components: Utilities
>    Affects Versions: 2.4
>         Environment: all environments
>            Reporter: Bernd Hopp
>            Priority: Minor
>              Labels: newbie, performance
>             Fix For: 2.5
>
>   Original Estimate: 12h
>  Remaining Estimate: 12h
>
> In a lot of places, we allocate new buffers dynamically via new byte[]. This is a performance
drawback since many of these allocations could be avoided if we would use threadlocal buffers
that can be reused. For example, consider the following code from IOUtils.java, ln 2177:
> return copyLarge(input, output, inputOffset, length, new byte[DEFAULT_BUFFER_SIZE]);
> This code allocates new memory for every copy-process, that is not used outside of the
method and could easily and safely reused, as long as is is thread-local. So instead of allocating
new memory, a new utility-class could provide a thread-local bytearray like this:
> byte[] buffer = ThreadLocalByteArray.ofSize(DEFAULT_BUFFER_SIZE);
> return copyLarge(input, output, inputOffset, length, buffer);
> I have not measured the performance-benefits yet, but I would expect them to be significant,
especially when the streams itself are not the performance bottleneck. 
> Git PR is at https://github.com/apache/commons-io/pull/6/files



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message