hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brice Arnould <br...@vleu.net>
Subject Re: using TreeMaps in JobTracker
Date Tue, 10 Jun 2008 12:00:49 GMT
Devaraj Das a écrit :
>> I suggest two changes :
>> 1- Make JobTracker.getTasksToKill(taskTracker) return a 
>> Collection, and make killJobIds that Collection, removing the 
>> need for copying its content into the killList
>> 2- Change the type of killJobIds from Set to ArrayList, since 
>> anyway it cannot contain duplicate elements, because its 
>> element are extracted from another set.
>>
> getTasksToKill returns two things in one list - the set of jobIDs the tasks
> of which should be killed and the set of taskIDs that should be killed. Note
> that the set of tasks is most likely just those for each of which another
> attempt has completed successfully. A tasktracker might be running other
> tasks of the same job and we don't want to kill those. The tasks for the
> jobs in killJobIDs are not expanded. This list is then copied to the
> "actions" list that contains everything the tasktracker should consider. So
> maybe the last copy can be avoided by having the JT store a reference to
> that action list and directly writing that out. But there are some other
> things to be taken care of like using arrays instead of lists ....
You're right. Thanks for the explanation ^_^

>> Yes. What I suggest is to make the "synchronized areas" 
>> smaller using concurrent containers and then to use a 
>> ThreadPool to answer heartbeats.
>> If you think that it is possible, I'll try to do it.
> This is definitely useful!
Cool ! I'll wait for #3412 to be reviewed (to avoid building a new patch 
on top of another patch that probably will change during the review 
process) then I'll do it

Brice

Mime
View raw message