hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4808) Blacklisting of TaskTrackers should take into account the user-ID
Date Tue, 09 Dec 2008 11:04:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654764#action_12654764
] 

Devaraj Das commented on HADOOP-4808:
-------------------------------------

A user can write malicious code that deterministically fails on a bunch of trackers but yet
the jobs succeed. Hence those trackers would get blacklisted for a long duration across all
jobs, after a certain number of such job runs. The malicious user can effectively hijack 50%
of the cluster (that's the limit after which trackers are not blacklisted across jobs). 
This can be improved by keeping an eye on how many users (user-IDs) blacklist a given tracker
across jobs and when this number (of users) cross a certain we mark that tracker blacklisted
across all jobs across all users. 
Thoughts?

> Blacklisting of TaskTrackers should take into account the user-ID
> -----------------------------------------------------------------
>
>                 Key: HADOOP-4808
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4808
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.20.0
>            Reporter: Devaraj Das
>
> With HADOOP-4305, it is possible to blacklist TaskTrackers across jobs. It might make
sense to also take into account the users whose tasks are being run on the TaskTrackers, and
use it in the blacklisting strategy.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message