commons-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bernd Eckenfels (JIRA)" <>
Subject [jira] [Commented] (IO-429) ByteArrayOutputStream can overflow
Date Wed, 12 Mar 2014 20:44:48 GMT


Bernd Eckenfels commented on IO-429:

The current code is nicely reusing ByteArrayOutputStream as well as largeCopy() from IOUtils.
There would be the option to implement it much more directly (using a gatherlist approach
and size limited buffers), which would have a number of advantages:

- reduced array copies (each byte at most once)
- controlled array resizing (for example 2 step linear growth)
- flatter stacks
- using some available() hints

But the code would be more complicated. Something like (pseudocode):

buf = new byte[minsize];
  c =,offset,bug.length-offset);
  read += c;
if (eof) return Arrays.copy(buf,read);
gatherlist = new ArrayList<byte[])(); // max 2047 1mb fragments
while(true) {
  buf= new byte[normsize]; 
  c = in.readFully(buf);
  if read > MAX_INT
  if (eof) return arrayFrom(gatherlist, buf);

If you think it is worth to have this kind of optimized implementation I can provide a working

But actually ByteBuffer would be much nicer as a return type.

> ByteArrayOutputStream can overflow
> ----------------------------------
>                 Key: IO-429
>                 URL:
>             Project: Commons IO
>          Issue Type: Bug
>          Components: Utilities
>            Reporter: Fabian Lange
> There are many places involved in the problem, and a good fix might be problematic performance
> For example:
> IOUtils.toByteArray(InputStream input) invoked with a Stream which feeds more than Integer.MAX_VALUE
bytes will either crash with NegativeArraySizeException or maybe worse overflow in such a
way that it returns fine (but only with partial data)
> The ByteArrayOutputStream will happily consume the full stream but "int count" will overflow.
At some point then toByteArray is invoked which will do like new byte[count].
> maybe "needNewBuffer" can throw the IllegalArgumentException, as it gets  the count and
could check for the overflow.

This message was sent by Atlassian JIRA

View raw message