Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <317978684.1228820684498.JavaMail.jira@brutus>
Date: Tue, 9 Dec 2008 03:04:44 -0800 (PST)
From: "Devaraj Das (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-4808) Blacklisting of TaskTrackers should
 take into account the user-ID
In-Reply-To: <286956328.1228820084175.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654764#action_12654764 ] 

Devaraj Das commented on HADOOP-4808:
-------------------------------------

A user can write malicious code that deterministically fails on a bunch of trackers but yet the jobs succeed. Hence those trackers would get blacklisted for a long duration across all jobs, after a certain number of such job runs. The malicious user can effectively hijack 50% of the cluster (that's the limit after which trackers are not blacklisted across jobs). 
This can be improved by keeping an eye on how many users (user-IDs) blacklist a given tracker across jobs and when this number (of users) cross a certain we mark that tracker blacklisted across all jobs across all users. 
Thoughts?

> Blacklisting of TaskTrackers should take into account the user-ID
> -----------------------------------------------------------------
>
>                 Key: HADOOP-4808
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4808
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.20.0
>            Reporter: Devaraj Das
>
> With HADOOP-4305, it is possible to blacklist TaskTrackers across jobs. It might make sense to also take into account the users whose tasks are being run on the TaskTrackers, and use it in the blacklisting strategy.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.