hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6000) native-task: Simplify ByteBufferDataReader/Writer
Date Sat, 13 Sep 2014 13:47:54 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132732#comment-14132732

Hudson commented on MAPREDUCE-6000:

FAILURE: Integrated in Hadoop-Mapreduce-trunk #1895 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1895/])
MAPREDUCE-6000. native-task: Simplify ByteBufferDataReader/Writer. Contributed by Todd Lipcon.
(todd: rev 663518032449940f136ae8a9f149ed453837a8d8)
* hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/java/org/apache/hadoop/mapred/nativetask/buffer/ByteBufferDataReader.java
* hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/java/org/apache/hadoop/mapred/nativetask/buffer/ByteBufferDataWriter.java
* hadoop-mapreduce-project/CHANGES.MAPREDUCE-2841.txt
* hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/test/java/org/apache/hadoop/mapred/nativetask/buffer/TestByteBufferReadWrite.java

> native-task: Simplify ByteBufferDataReader/Writer
> -------------------------------------------------
>                 Key: MAPREDUCE-6000
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6000
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: task
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Minor
>         Attachments: mapreduce-6000.txt, mapreduce-6000.txt
> The ByteBufferDataReader and ByteBufferDataWriter class are more complex than necessary:
> - several methods related to reading/writing strings and char arrays are implemented
but never used by the native task code. Given that the use case for these classes is limited
to serializing binary data to/from the native code, it seems unlikely people will want to
use these methods in any performance-critical space. So, let's do simpler implementations
that are less likely to be buggy, even if they're slightly less performant.
> - methods like readLine() are even less likely to be used. Since it's a complex implementation,
let's just throw UnsupportedOperationException
> - in the test case, we can use Mockito to shorten the amount of new code

This message was sent by Atlassian JIRA

View raw message