incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Massimiliano Mattetti (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BLUR-449) The commitOrAbortJob method of the BlurOutputCommitter class ends up always with the warning "TaskAttempt JobID [{0}] does not match JobContext JobId [{1}]"
Date Wed, 06 Apr 2016 08:25:25 GMT
Massimiliano Mattetti created BLUR-449:
------------------------------------------

             Summary: The commitOrAbortJob method of the BlurOutputCommitter class ends up
always with the warning "TaskAttempt JobID [{0}] does not match JobContext JobId [{1}]"
                 Key: BLUR-449
                 URL: https://issues.apache.org/jira/browse/BLUR-449
             Project: Apache Blur
          Issue Type: Bug
          Components: Blur MapReduce
    Affects Versions: 0.3.0
         Environment: Spark 1.6.0 + Hadoop 2.6.4
            Reporter: Massimiliano Mattetti


The equals in this if statement: 

if (taskAttemptID.getJobID().equals(jobContext.getJobID())) 

is always returning false. The problem is similar to the one described here https://issues.apache.org/jira/browse/MAPREDUCE-1842.
The TaskAttemptID.forName method instantiate a new TaskAttemptID object whose type comes from
the older API package org.apache.hadoop.mapred. Since the JobContext contains a JobID form
the newer package org.apache.hadoop.mapreduce and the equals method shared by the two JobIDs
uses this.getClass() == that.getClass, the equals between the two objects will always return
false. Comparing the JobID Strings instead of the objects is a possible workaround otherwise
another possibility would be to instantiate a org.apache.hadoop.mapreduce.TaskAttemptID object
using the one retuned from the TaskAttemptID.forName method.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message