hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6381) WritableUtils::*VLong utilities should be available for byte arrays
Date Wed, 18 Nov 2009 21:24:39 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12779676#action_12779676
] 

Chris Douglas commented on HADOOP-6381:
---------------------------------------

bq. Doesn't it avoid the need to construct a ByteArrayInputStream?

Yes, but if one has a DataInputBuffer for the purpose, then I wouldn't expect it to be too
expensive. Given a key type with two vint params, to sort by the second:
{code}
final DataInputBuffer di = new DataInputBuffer(0);

int compare(byte[] b1, int s1, int l1, byte[] b2, int s2, int l2) {
  di.reset(b1, s1, l1);
  final int v1 = WritableUtils.readVInt(di);
  final int x1 = WritableUtils.decodeVIntSize(b1[s1]);
  // second param starts at b[s1 + v1 + x1]
...
{code}

Which isn't too expensive or inelegant. Thread safety and a memory leak from the DIB reference
can both be handled, again, without imposing too high a penalty, even for an inner loop. It
just struck me as a lot of ceremony.

> WritableUtils::*VLong utilities should be available for byte arrays
> -------------------------------------------------------------------
>
>                 Key: HADOOP-6381
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6381
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: util
>            Reporter: Chris Douglas
>            Priority: Minor
>         Attachments: C6381-0.patch, C6381-1.patch, C6381-2.patch
>
>
> Particularly when working with raw bytes in Writables, it is often useful to have versions
of the vint utility functions for byte arrays.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message