Mailing-List: contact issues-help@commons.apache.org; run by ezmlm
Precedence: bulk
Reply-To: issues@commons.apache.org
Date: Sun, 8 Feb 2015 13:04:34 +0000 (UTC)
From: "Sebb (JIRA)" <jira@apache.org>
To: issues@commons.apache.org
Message-ID: <JIRA.12771961.1422949842000.294511.1423400674647@Atlassian.JIRA>
In-Reply-To: <JIRA.12771961.1422949842000@Atlassian.JIRA>
References: <JIRA.12771961.1422949842000@Atlassian.JIRA>
 <JIRA.12771961.1422949842797@arcas>
Subject: [jira] [Commented] (IO-468) Avoid allocating memory for method
 internal buffers, use threadlocal memory instead
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/IO-468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311308#comment-14311308 ] 

Sebb commented on IO-468:
-------------------------

What version of Java did you use for compile/test?

As I already wrote, I don't dispute that TL performs better for single-threaded tests when using larger buffers.
The problem is the additional memory which is held for potentially much longer periods.

The PerfTest.java runnable includes setup costs (random etc) which should probably be done in advance.
Also there is no need to clone the updated IO source - the test code can implement its own TL buffering and provide that to the lower-level copy method.
For additional points, it might be worth comparing the performance where the user provides a fixed buffer.


> Avoid allocating memory for method internal buffers, use threadlocal memory instead
> -----------------------------------------------------------------------------------
>
>                 Key: IO-468
>                 URL: https://issues.apache.org/jira/browse/IO-468
>             Project: Commons IO
>          Issue Type: Improvement
>          Components: Utilities
>    Affects Versions: 2.4
>         Environment: all environments
>            Reporter: Bernd Hopp
>            Priority: Minor
>              Labels: newbie, performance
>             Fix For: 2.5
>
>         Attachments: PerfTest.java, monitoring_with_threadlocals.png, monitoring_without_threadlocals.png, performancetest.ods
>
>   Original Estimate: 12h
>  Remaining Estimate: 12h
>
> In a lot of places, we allocate new buffers dynamically via new byte[]. This is a performance drawback since many of these allocations could be avoided if we would use threadlocal buffers that can be reused. For example, consider the following code from IOUtils.java, ln 2177:
> return copyLarge(input, output, inputOffset, length, new byte[DEFAULT_BUFFER_SIZE]);
> This code allocates new memory for every copy-process, that is not used outside of the method and could easily and safely reused, as long as is is thread-local. So instead of allocating new memory, a new utility-class could provide a thread-local bytearray like this:
> byte[] buffer = ThreadLocalByteArray.ofSize(DEFAULT_BUFFER_SIZE);
> return copyLarge(input, output, inputOffset, length, buffer);
> I have not measured the performance-benefits yet, but I would expect them to be significant, especially when the streams itself are not the performance bottleneck. 
> Git PR is at https://github.com/apache/commons-io/pull/6/files


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)