hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Kimball <aa...@cloudera.com>
Subject Re: BytesWriteable, why no set(byte[])?
Date Sun, 05 Jul 2009 06:32:10 GMT
It does an array copy because there is no contract that you won't change
those bytes out from under the object. The Writable objects are assumed to
be wholely self-contained.

You could add a patch to Hadoop that does not perform the capacity increase
and provides the set(byte[]) method. You could also implement your own
MyBytesWritable class yourself, copying 95% of the code from the existing

- Aaron

On Thu, Jul 2, 2009 at 6:05 PM, william kinney <william.kinney@gmail.com>wrote:

> Hi,
> Just wondering why there is no set(byte[]) for BytesWriteable, in the
> similar fashion of the constructor? And why does the set(byte[], int,
> int) do a System.arraycopy()?
> My real issue is that in writing my own RecordReader, I am trying to
> set the byte[] for a BytesWriteable that is passed in. This works fine
> for set(byte[], int, int), but then doing a get() on it later in the
> Map phase I get unwanted buffered data that appears to because the
> set(byte[], int, int) adds buffered capacity of the magnitude 3/2 *
> size.
> Thanks for the help,
> Will

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message