hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-4149) JobQueueJobInProgressListener.jobUpdated() might not work as expected
Date Wed, 10 Sep 2008 14:34:44 GMT
JobQueueJobInProgressListener.jobUpdated() might not work as expected
---------------------------------------------------------------------

                 Key: HADOOP-4149
                 URL: https://issues.apache.org/jira/browse/HADOOP-4149
             Project: Hadoop Core
          Issue Type: Bug
          Components: mapred
            Reporter: Amar Kamat


{{JobQueueJobInProgressListener}} uses a {{TreeSet}} to store the sorted collection of {{JobInProgress}}
objects. The comparator used to sort the JIPs follow the following order
- priority (>=)
- start time (<=)
- job id [jt-identifier, job-index] (<=)

If any JIP object is changed w.r.t priority or start-time, then the TreeSet will be inconsistent.
Hence doing  a delete might not work. Consider the following
1) jobs are submitted in the following order 
||number||jobid||priority||
|1|j1|NORMAL|
|2|j2|LOW|
|3|j3|NORMAL|

2) The sorted collection will be in the order : {{j1,j3,j2}}

3) If job3's priority is changed to LOW then the collection wont change but delete will bail
out on j1 itself as the comparator will return a -ve number. TreeSet uses the comparator both
for sorting and deleting. If  i indicates the index in the collection and obj represents the
object under consideration, then looks like TreeSet.remove(obj) follows something like  :
- continue to search if the compare(i, obj) is -ve
- bail out if the compare(i, obj) is +ve
- delete the obj of compare(i,obj) == 0

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message