commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andreas Niedermeier (JIRA)" <>
Subject [jira] Commented: (IO-218) Introduce new filter input stream with replacement facilities
Date Tue, 18 Jan 2011 09:02:46 GMT


Andreas Niedermeier commented on IO-218:

Hi, first of all great work, but while using this class I sometimes experienced some problems.
While replacing sometimes {{0}}-bytes would appear in my result. I found out that it only
seems to happen when {{replacementTo}} is larger than {{replacementFrom}}. Doing some debugging
I realized that in {{replaceWithExpand()}} the {{ByteBuffer}}s limit would be set to a very
large value (small input and pretty large buffer used) resulting in much more data in the
result than expected (which was all filled with zeros).
When there is plenty of space in the buffer {{totalUnread}} will be a large negative number
resulting in a negative {{unreadBufferSize}} (lines 509 and 510).
This {{unreadBufferSize}} will lead to a very large {{moveLength}} on line 528 ({{int moveLength
= data.remaining() - unreadBufferSize;}}) which is added to {{data.limit()}} on line 538 resulting
in a buffer pretending to contain far more data than it really does.

If the size of the array used in the Test (line 218) is increased from 3 to 4 {{bufferOverflow()}}
will fail. Increasing it to 6 will also break {{toClashesFrom()}} and increasing it to 8 will
also cause {{toLongerThanFrom()}} to fail (for the described reason).
Reducing the size to smaller than 3 will cause an infinite loop while {{toClashesFrom()}}.

I've made the following two changes to ReplaceFilterInputStream:
* on line 517 I inserted
if (unreadBufferSize < 0) {
    unreadBufferSize = 0;
so that a negative {{unreadBufferSize}} won't increase {{moveLength}}
* changed line 538 to
data.limit(Math.min(data.limit() + diff, data.capacity()));
because I assumed that the limit is only increasing by the length difference of the replacements
and not the remaining data in the buffer

I'm not sure if it does all it shoud but so far it seems to work for me and the tests succeed
with buffer sizes > 2.

> Introduce new filter input stream with replacement facilities
> -------------------------------------------------------------
>                 Key: IO-218
>                 URL:
>             Project: Commons IO
>          Issue Type: Improvement
>          Components: Filters
>    Affects Versions: 1.4
>         Environment: all environments
>            Reporter: Denis Zhdanov
>         Attachments:,
>   Original Estimate: 120h
>  Remaining Estimate: 120h
> It seems convenient to have a FilterInputStream that allows to apply predefined repalcement
rules against the read data. 
> For example we may want to configure the following replacements:
> {noformat}
> {1, 2} -> {7, 8}
> {1} -> {9}
> {3, 2} -> {}
> {noformat}
> and apply them to the input like
> {noformat}
> {4, 3, 2, 1, 2, 1, 3}
> {noformat}
> in order to get a result like
> {noformat}
> {4, 7, 8, 9, 3}
> {noformat}
> I created the class that allows to do that and attached it to this ticket. Unit test
class at junit4 format is attached as well.
> So, the task is to review the provided classes, consider if it's worth to add them to
commons-io distribution and perform the inclusion in the case of possible result.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message