hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6381) WritableUtils::*VLong utilities should be available for byte arrays
Date Wed, 18 Nov 2009 06:30:40 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12779329#action_12779329
] 

Chris Douglas commented on HADOOP-6381:
---------------------------------------

bq. decodeVint methods only return the integer and not the length consumed, so again they're
less than useful in a lot of circumstances.

That's what {{WritableUtils::decodeVIntSize}} does.

bq. I am a bit concerned with the code duplication this may introduce.

I wrote the read to use with a RawComparator. For e.g. tuples that vint encode the length
of each parameter, it's easier to skip over bytes using the form this introduces. One could
keep a DataInputBuffer with the comparator, but that either requires an allocation or has
threading issues. To be fair, WritableComparator isn't threadsafe either, but adding a ThreadLocal,
etc. seemed like an unnecessary evasion for a byte-oriented decode.

We're committed to the vint format written from WritableUtils... it's not clear what problems
duplicating this code would cause, unless the new code is incorrect. It's not an efficiency
improvement; it's just a helpful API when writing RawComparators. It seemed like a natural
form to have for working with vints, but I'm wholly ambivalent about closing this as "won't
fix".

> WritableUtils::*VLong utilities should be available for byte arrays
> -------------------------------------------------------------------
>
>                 Key: HADOOP-6381
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6381
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: util
>            Reporter: Chris Douglas
>            Priority: Minor
>         Attachments: C6381-0.patch, C6381-1.patch, C6381-2.patch
>
>
> Particularly when working with raw bytes in Writables, it is often useful to have versions
of the vint utility functions for byte arrays.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message