Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D2DF3101C2 for ; Tue, 30 Jul 2013 18:39:19 +0000 (UTC) Received: (qmail 65711 invoked by uid 500); 30 Jul 2013 18:39:18 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 65659 invoked by uid 500); 30 Jul 2013 18:39:17 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 65648 invoked by uid 99); 30 Jul 2013 18:39:17 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 30 Jul 2013 18:39:17 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of pandeesh@gmail.com designates 209.85.216.182 as permitted sender) Received: from [209.85.216.182] (HELO mail-qc0-f182.google.com) (209.85.216.182) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 30 Jul 2013 18:39:13 +0000 Received: by mail-qc0-f182.google.com with SMTP id c11so2714403qcv.13 for ; Tue, 30 Jul 2013 11:38:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=bcmJayQdqKEHZttX4GykxML0+7tip9fZ77rCuBWWo+c=; b=VAOWXCdcz+eM1jmRzacg+AB8PCYMMfA5K/b4YDuP2FhBBA0RnyqXIRAOGY7IKhrKYM zrk8PyW3YCctlYTjW093tzspgX1NGfaLtoP2WSJHfyoimbYCMKL7VVuHJL8gX3ffvpkw zlxQwBARGH/q8KklrRQZoVvoPdLJ3fhmYt268JTJ1yWuIobnZTZhpSWBjsWMiecmuhjo 9+u/TAOtf6nqu9XL/T6WetmcjE04VKqcq1Q9EDHRiSR39LT91FRdWYVxistr+S4S6lqK pWgoHcIkRhlodB1NAiLJcZlgMBttIB2i/TkItpFu1RNPbmMIQQamIzl5A34VlAkjg9ol h6tw== MIME-Version: 1.0 X-Received: by 10.49.72.201 with SMTP id f9mr45861126qev.85.1375209532410; Tue, 30 Jul 2013 11:38:52 -0700 (PDT) Received: by 10.49.49.201 with HTTP; Tue, 30 Jul 2013 11:38:52 -0700 (PDT) Received: by 10.49.49.201 with HTTP; Tue, 30 Jul 2013 11:38:52 -0700 (PDT) In-Reply-To: References: <289D19EB-F946-4604-A0FE-5233D9865AF1@apache.org> Date: Wed, 31 Jul 2013 00:08:52 +0530 Message-ID: Subject: Re: Prevent users from killing each other's jobs From: pandees waran To: user@hive.apache.org Content-Type: multipart/alternative; boundary=047d7b677be4abd35604e2beebc2 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b677be4abd35604e2beebc2 Content-Type: text/plain; charset=UTF-8 Hi Mikhail, Could you please explain how we can track all the kill requests for a job? Is there any feature available in hadoop stack for this? Or do we need to track this in OS layer by capturing the signals? Thanks, Pandeesh On Jul 31, 2013 12:03 AM, "Mikhail Antonov" wrote: > In addition to using job's ACLs you could have more brutal schema. Track > all requests to kill the jobs, and if any request is coming from the user > who should't be trying to kill this particular job, then ssh from the > script to his client machine and forcibly reboot it :) > > > 2013/7/30 Edward Capriolo > >> Honestly tell your users to stop being jerks. People know if they kill my >> query there is going to be hell to pay :) >> >> >> On Tue, Jul 30, 2013 at 2:25 PM, Vinod Kumar Vavilapalli < >> vinodkv@apache.org> wrote: >> >>> >>> You need to set up Job ACLs. See >>> http://hadoop.apache.org/docs/stable/mapred_tutorial.html#Job+Authorization >>> . >>> >>> It is a per job configuration, you can provide with defaults. If the job >>> owner wishes to give others access, he/she can do so. >>> >>> Thanks, >>> +Vinod Kumar Vavilapalli >>> Hortonworks Inc. >>> http://hortonworks.com/ >>> >>> On Jul 30, 2013, at 11:21 AM, Murat Odabasi wrote: >>> >>> Hi there, >>> >>> I am trying to introduce some sort of security to prevent different >>> people using the cluster from interfering with each other's jobs. >>> >>> Following the instructions at >>> http://hadoop.apache.org/docs/stable/cluster_setup.html and >>> >>> https://www.inkling.com/read/hadoop-definitive-guide-tom-white-3rd/chapter-9/security >>> , this is what I put in my mapred-site.xml: >>> >>> >>> mapred.task.tracker.task-controller >>> org.apache.hadoop.mapred.LinuxTaskController >>> >>> >>> >>> mapred.acls.enabled >>> true >>> >>> >>> I can see the configuration parameters in the job configuration when I >>> run a hive query, but the users are still able to kill each other's >>> jobs. >>> >>> Any ideas about what I may be missing? >>> Any alternative approaches I can adopt? >>> >>> Thanks. >>> >>> >>> >> > > > -- > Thanks, > Michael Antonov > --047d7b677be4abd35604e2beebc2 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Hi Mikhail,

Could you please explain how we can track all the kill reque= sts for a job?=C2=A0 Is there any feature available in hadoop stack for thi= s? Or do we need to track this in OS layer by capturing the signals?

Thanks,
Pandeesh

On Jul 31, 2013 12:03 AM, "Mikhail Antonov&= quot; <olorinbant@gmail.com&= gt; wrote:
In addition to using job's ACLs you could have more br= utal schema. Track all requests to kill the jobs, and if any request is com= ing from the user who should't be trying to kill this particular job, t= hen ssh from the script to his client machine and forcibly reboot it :)


2013/7/30 Edw= ard Capriolo <edlinuxguru@gmail.com>
Honestly tell your users to stop being jerks. People know = if they kill my query there is going to be hell to pay :)


On Tue, Jul 30, 2013 at 2:25 PM, Vinod Kumar Vavilapalli = <vinodkv@apache.= org> wrote:

It is a per job configuration, you can provide with def= aults. If the job owner wishes to give others access, he/she can do so.

Thanks,
+Vinod Kumar Vavilapalli
Hortonworks Inc.<= br>http://hortonworks= .com/

On Jul 30, 2013, at 11:21 AM, Murat Odabasi wrote:

<= blockquote type=3D"cite">
Hi there,

I am trying to introduce som= e sort of security to prevent different
people using the cluster from in= terfering with each other's jobs.

Following the instructions at
http://hadoop.apache.org/do= cs/stable/cluster_setup.html and
https://www.inkling.com/read/hadoop-definitive-guide-tom-white-3rd/ch= apter-9/security
, this is what I put in my mapred-site.xml:

<property>
=C2= =A0<name>mapred.task.tracker.task-controller</name>
=C2=A0&= lt;value>org.apache.hadoop.mapred.LinuxTaskController</value>
&= lt;/property>

<property>
=C2=A0<name>mapred.acls.enabled</name>=
=C2=A0<value>true</value>
</property>

I ca= n see the configuration parameters in the job configuration when I
run a= hive query, but the users are still able to kill each other's
jobs.

Any ideas about what I may be missing?
Any alternative appr= oaches I can adopt?

Thanks.





--
= Thanks,
Michael Antonov --047d7b677be4abd35604e2beebc2--