hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-353) Allow shuffle read and connection timeouts to be configurable
Date Tue, 01 Dec 2009 10:01:21 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784151#action_12784151
] 

Amareshwari Sriramadasu commented on MAPREDUCE-353:
---------------------------------------------------

test-patch for Y!20 patch :
{noformat}
     [exec] -1 overall.
     [exec]
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec]
     [exec]     -1 tests included.  The patch doesn't appear to include any new or modified
tests.
     [exec]                         Please justify why no tests are needed for this patch.
     [exec]
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec]
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler
warnings.
     [exec]
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
{noformat}
Ttested manually by setting small values for these timeouts and saw getting the socketTimeoutException
as in https://issues.apache.org/jira/browse/MAPREDUCE-353?focusedCommentId=12730697&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12730697

All unit tests passed except TestHdfsProxy

> Allow shuffle read and connection timeouts to be configurable
> -------------------------------------------------------------
>
>                 Key: MAPREDUCE-353
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-353
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>    Affects Versions: 0.21.0
>            Reporter: Arun C Murthy
>            Assignee: Ravi Gummadi
>             Fix For: 0.21.0
>
>         Attachments: MR-353.patch, MR-353.v1.patch, patch-353-ydist.txt
>
>
> It would be good for latency-sensitive applications to tune the shuffle read/connection
timeouts... in fact this made a huge difference to terasort since we were seeing individual
shuffles stuck for upwards of 60s and had to have a very small read timeout.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message