hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3398) ReduceTask::closestPowerOf2 is inefficient
Date Fri, 16 May 2008 04:42:55 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12597350#action_12597350
] 

Amar Kamat commented on HADOOP-3398:
------------------------------------

Code looks fine. The values are same for both the implementations. Some comments as to what
is done could be useful. I compared the two implementations 
||input||num-iterations||before (milli)||after (milli)||
|10000|1000000|642|19|
|5000|1000000|78|9|
|2500|1000000|74|5|
|1000|1000000|69|3|
|500|1000000|51|1|
|300|1000000|49|1|
Assuming that the map runtime can vary from 300sec to 10,000 secs and the maximum number of
times {{getClosestPowerOf2()}} will be invoked is 1,000,000.

> ReduceTask::closestPowerOf2 is inefficient
> ------------------------------------------
>
>                 Key: HADOOP-3398
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3398
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>            Priority: Trivial
>             Fix For: 0.18.0
>
>         Attachments: 3398-0.patch, 3398-1.patch
>
>
> ReduceTask computes the "closest power of 2" using loops
> {code}
> private static int getClosestPowerOf2(int value) {
>   int power = 0;
>   int approx = 1;
>   while (approx < value) {
>     ++power;
>     approx = (approx << 1);
>   }
>   if ((value - (approx >> 1)) < (approx - value)) {
>     --power;
>   }
>   return power;
> }
> {code}
> This could be improved.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message