hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-825) JobClient completion poll interval of 5s causes slow tests in local mode
Date Wed, 05 Aug 2009 03:51:14 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739328#action_12739328

Vinod K V commented on MAPREDUCE-825:

Now that we are doing this, shouldn't we also do something for the hard-coded sleep interval
in monitorAndPrintJob() (JobClient.java +1294)? That might help in some test-cases too..

Regarding tests, you can change any of the current test cases to use a lower value for the
introduced configuration and verify manually that the test time has decreased. I could see
TestTaskFail using three instances of waitForCompletion() in a single test, may be you can
change that test-case and see how it goes.

> JobClient completion poll interval of 5s causes slow tests in local mode
> ------------------------------------------------------------------------
>                 Key: MAPREDUCE-825
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-825
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>            Priority: Minor
>         Attachments: completion-poll-interval.patch
> The JobClient.NetworkedJob.waitForCompletion() method polls for job completion every
5 seconds. When running a set of short tests in pseudo-distributed mode, this is unnecessarily
slow and causes lots of wasted time. When bandwidth is not scarce, setting the poll interval
to 100 ms results in a 4x speedup in some tests.  This interval should be parametrized to
allow users to control the interval for testing purposes.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message