hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5744) Job hangs because RMContainerAllocator$AssignedRequests.preemptReduce() violates the comparator contract
Date Thu, 06 Feb 2014 22:27:25 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13893902#comment-13893902
] 

Hadoop QA commented on MAPREDUCE-5744:
--------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12627473/MAPREDUCE-5744.v02.patch
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

    {color:green}+1 eclipse:eclipse{color}.  The patch built with eclipse:eclipse.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version
1.3.9) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 core tests{color}.  The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app.

    {color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4348//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4348//console

This message is automatically generated.

> Job hangs because RMContainerAllocator$AssignedRequests.preemptReduce() violates the
comparator contract
> --------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5744
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5744
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.0.5-alpha
>            Reporter: Sangjin Lee
>            Assignee: Gera Shegalov
>            Priority: Blocker
>         Attachments: MAPREDUCE-5744.v02.patch
>
>
> We ran into a situation where tasks are not getting assigned because RMContainerAllocator$AssignedRequests.preemptReduce()
fails repeatedly with the following exception:
> {code}
> 2014-02-06 16:43:45,183 ERROR [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator:
ERROR IN CONTACTING RM.
> java.lang.IllegalArgumentException: Comparison method violates its general contract!
>      at java.util.TimSort.mergeLo(TimSort.java:747)
>      at java.util.TimSort.mergeAt(TimSort.java:483)
>      at java.util.TimSort.mergeCollapse(TimSort.java:408)
>      at java.util.TimSort.sort(TimSort.java:214)
>      at java.util.TimSort.sort(TimSort.java:173)
>      at java.util.Arrays.sort(Arrays.java:659)
>      at java.util.Collections.sort(Collections.java:217)
>      at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator$AssignedRequests.preemptReduce(RMContainerAllocator.java:1106)
>      at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.preemptReducesIfNeeded(RMContainerAllocator.java:416)
>      at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.heartbeat(RMContainerAllocator.java:230)
>      at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator$1.run(RMCommunicator.java:252)
>      at java.lang.Thread.run(Thread.java:744)
> {code}
> It is because the comparator that's defined in this method does not abide by the contract,
specifically if p == 0.
> Comparator.compare(): http://docs.oracle.com/javase/7/docs/api/java/util/Comparator.html#compare(T,
T)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message