hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mahadev konar (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3326) ReduceTask should not sleep for 200 ms while waiting for merge to finish
Date Wed, 04 Jun 2008 19:06:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12602417#action_12602417
] 

Mahadev konar commented on HADOOP-3326:
---------------------------------------

the interrupt seems wrong. the lines 

    //copiers are done, interrupt the waiting merge threads 
        localFSMergerThread.interrupt();
        inMemFSMergeThread.interrupt();
        

interrupt both the threads when the copier is done. In that case the work that these threads
were doing will not be completed. Menaing theymight still be merging in memory or from local
disk and data might end up disappearding if they do not complete the work they are doing.
Is that correct?


> ReduceTask should not sleep for 200 ms while waiting for merge to finish
> ------------------------------------------------------------------------
>
>                 Key: HADOOP-3326
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3326
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Sharad Agarwal
>             Fix For: 0.18.0
>
>         Attachments: 3326_1.patch, 3326_2.patch
>
>
> Currently the merge code in Reduce task does:
> {code}
>             // Wait for the on-disk merge to complete
>             while (localFSMergeInProgress) {
>               Thread.sleep(200);
>             }
>             
>             //wait for an ongoing merge (if it is in flight) to complete
>             while (mergeInProgress) {
>               Thread.sleep(200);
>             }
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message