hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Binglin Chang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6067) native-task: spilled records counter is incorrect
Date Fri, 05 Sep 2014 06:15:26 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122496#comment-14122496

Binglin Chang commented on MAPREDUCE-6067:

Thanks for the review and tests, guys. Will commit soon. 

> native-task: spilled records counter is incorrect
> -------------------------------------------------
>                 Key: MAPREDUCE-6067
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6067
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: task
>            Reporter: Todd Lipcon
>            Assignee: Binglin Chang
>         Attachments: MAPREDUCE-6067.v1.patch, MAPREDUCE-6067.v2.patch, MAPREDUCE-6067.v3.patch,
MAPREDUCE-6067.v4.patch, MAPREDUCE-6067.v5.patch, native-counters.html, trunk-counters.html
> After running a terasort, I see the spilled records counter at 5028651606, which is about
half what I expected to see. Using the non-native collector I see the expected count of 10000000000.
It seems the correct number of records were indeed spilled, because the job's output record
count is correct.

This message was sent by Atlassian JIRA

View raw message