hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clément MAHTIEU (JIRA) <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-6034) JobControl: polling interval should not be hard coded
Date Wed, 13 Aug 2014 13:37:11 GMT
Clément MAHTIEU created MAPREDUCE-6034:

             Summary: JobControl: polling interval should not be hard coded
                 Key: MAPREDUCE-6034
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6034
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: client
            Reporter: Clément MAHTIEU

The JobControl mechanism relies on a polling mechanism which is implemented in JobControl.run().

The polling period is currently hard coded to 5s in the run() method making it almost impossible
to tune. However, I believe that it will be good to let the users set this value. Some applications
could want it in production and it would speed up the local mode and integration tests a lot.
In my current project, nearly half of the time of the test suite is spend in this single line.
Pig had the same issue, and decided to "crazily" hack the JobControl class rather than fixing
it in the upstream (see PIG-2702).

AFAIK it could be easily done by providing a new ctor or a new setter. This change would not
break the compatibility. I can provide a patch if this change seems reasonable. 

This message was sent by Atlassian JIRA

View raw message