Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 2BA59200D4F for ; Wed, 6 Dec 2017 19:50:07 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 2A2EF160C0A; Wed, 6 Dec 2017 18:50:07 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 4B8C3160BF3 for ; Wed, 6 Dec 2017 19:50:06 +0100 (CET) Received: (qmail 47965 invoked by uid 500); 6 Dec 2017 18:50:05 -0000 Mailing-List: contact user-help@impala.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@impala.apache.org Delivered-To: mailing list user@impala.apache.org Received: (qmail 47955 invoked by uid 99); 6 Dec 2017 18:50:05 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 Dec 2017 18:50:05 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id A91471805CC for ; Wed, 6 Dec 2017 18:50:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.379 X-Spam-Level: ** X-Spam-Status: No, score=2.379 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id cqmz1kRiAaGR for ; Wed, 6 Dec 2017 18:50:03 +0000 (UTC) Received: from mail-it0-f46.google.com (mail-it0-f46.google.com [209.85.214.46]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 06C065F244 for ; Wed, 6 Dec 2017 18:50:03 +0000 (UTC) Received: by mail-it0-f46.google.com with SMTP id x28so8762741ita.0 for ; Wed, 06 Dec 2017 10:50:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=/Fekk+TLPjbApz5oPUb31Rt/XYEiRfbYJvN4t7nYb1o=; b=Vz16XHtb/SlRV5IVoa2byWcUynw1nSb/Azl/E2NMMQIiXK9p/piG0svi8SZgqjyTi7 8iYYGz2H/gmJp00zkcR1QLbQf22kvrEvUxMxNl2rAW9+BkULE7TuYM9OXQaHVcIINrAG OSfEzJ1d2JkeV9qaJXt49LdQur+uF6QiNLFXP90rQE9lrxMmjBb+lTRiOBR6Y5Va+ujQ lEHtxzmfdGGRLiaE+dh2DMebk325kkbmY3Bpxe+xIUEyqBGtUqDGlcTpldz3svnQpgdA 6S9+OvXdNZ4yyNy3tLA35HMjnyNhkYM+CDY8BOaEvJ8UhzxgnYAN0QJRA4A6+dvB0Gen l0wQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=/Fekk+TLPjbApz5oPUb31Rt/XYEiRfbYJvN4t7nYb1o=; b=ao8EUJZEwmkJbZLeAOc+ebBumnsMaeJ7FfNTnYskWjUIb98bbBcHDARQ/seXKKlA8Y oe66Ihi7oKofXxxON4magQxEkrUEW/86hXJ8oEP3g8tozas9XFjc6GGsZMvpYK39CqrE Et7oXprtOujqJE4M4n1YqkiakgJJ5guBMw5k9RRUWTiZTlaP9nZ0n9kJ99lByhK8BRJ9 wramh+CL9vWDtKyBiZEfJEKbo0Wcur5HoBtM3bXZf/C1M0pD0BTpn31RpZypoTNZk+el EBZT1Trh/ENeAslRiQyw3FGWtoMRz8+7TCtrjmG48PS6lTzy3ReINbllZziT/E3dg9Le aHtA== X-Gm-Message-State: AJaThX4XkI7yywLFPRvia2QslyNuZf2QiMdhuqQopkweUw1tTzdB/Oy3 HDXZZge/6kSe2c9/0EiZekJbC9SLRSnZK0/uRVYj4A== X-Google-Smtp-Source: AGs4zMbUHVez2qg0JR6myhbxvJOf8OeTKvWdWL/n1Rey0J1fgSzSo9pxgxdf5e6K8f4zkVbsXuE/t8/aBS8qvBA+gFA= X-Received: by 10.107.174.222 with SMTP id n91mr33291471ioo.43.1512586201959; Wed, 06 Dec 2017 10:50:01 -0800 (PST) MIME-Version: 1.0 Received: by 10.79.79.202 with HTTP; Wed, 6 Dec 2017 10:49:41 -0800 (PST) In-Reply-To: References: From: Sunil Parmar Date: Wed, 6 Dec 2017 10:49:41 -0800 Message-ID: Subject: Re: INSERT query generated file permissions To: user@impala.apache.org Content-Type: multipart/alternative; boundary="001a1144683042973b055fb06b74" archived-at: Wed, 06 Dec 2017 18:50:07 -0000 --001a1144683042973b055fb06b74 Content-Type: text/plain; charset="UTF-8" Sure got it! One more related Impala question. The documentation for insert inherit permissions says. "If true, new directories created by INSERTs will inherit the permissions of their parent directories" Does it also applies to the files ? Sunil Parmar On Wed, Dec 6, 2017 at 10:27 AM, Jim Apple wrote: > That sounds right to me, but we try to keep this list focused on Apache > Impala and not any proprietary tools around it. You might have more success > here: http://community.cloudera.com/t5/Interactive- > Short-cycle-SQL/bd-p/Impala > > On Wed, Dec 6, 2017 at 10:13 AM, Sunil Parmar > wrote: > >> Thanks !! How to set insert_inherit_permissions from Cloudera manager / >> which security valve config to use and what to set in the value ? >> >> Impala Daemon Environment Advanced Configuration Snippet (Safety Valve) ? >> >> Sunil Parmar >> >> On Wed, Dec 6, 2017 at 6:50 AM, Petter von Dolwitz (Hem) < >> petter.von.dolwitz@gmail.com> wrote: >> >>> You can also explore HDFS ACLs to solve this problem. This is something >>> we use to give different user groups different privileges. See >>> https://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/ha >>> doop-hdfs/HdfsPermissionsGuide.html#ACLs_Shell_Commands >>> >>> Br, >>> Petter >>> >>> 2017-12-06 1:12 GMT+01:00 Jim Apple : >>> >>>> I'm not sure. Some things that come to mind: >>>> >>>> Did you try setting a umask for the impala user? >>>> >>>> Did you try setting the parent directory mode bits to 764 and then >>>> using the insert_inherit_permissions flag? >>>> >>>> On Tue, Dec 5, 2017 at 1:27 PM, Sunil Parmar >>>> wrote: >>>> >>>>> Environment CDH 5.12 >>>>> >>>>> When running INSERT query on the table all files are always owned by >>>>> user impala i.e. 744, all for impala read for everyone else . We have >>>>> externally running compaction process which needs to read/write/replace >>>>> this files. Is there a way to change this default behavior to have the file >>>>> permission different than 744 ? I'd prefer if it's 764 ( group read/write ) >>>>> so we can add the user to the same group as Impala who run the compaction >>>>> process. >>>>> >>>>> Sunil Parmar >>>>> >>>> >>>> >>> >> > --001a1144683042973b055fb06b74 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Sure got it!
One more related Impala qu= estion. The documentation for insert inherit permissions says.

"If true, new directories created by INSERTs will in= herit the permissions of their parent directories"

Does i= t also applies to the files ?


Sunil Parmar

On Wed, Dec 6, 2017 at 10:27 AM, Jim Apple <= span dir=3D"ltr"><jbapple@cloudera.com> wrote:
That sounds right to me, but we try to keep this lis= t focused on Apache Impala and not any proprietary tools around it. You mig= ht have more success here:=C2=A0http://communit= y.cloudera.com/t5/Interactive-Short-cycle-SQL/bd-p/Impala

On Wed, Dec 6, 2017 at 10:13 AM, Sunil Parmar <sunilosunil@gmail.com> wrote:
Thanks !! How to set insert_inherit_permissions from = Cloudera manager / which security valve config to use and what to set in th= e value ?

Impala Daemon Environment Advanced Configuration Snippet (= Safety Valve) ?

Sunil Parmar

On Wed, Dec 6, 2017 at 6:50 AM, Petter von D= olwitz (Hem) <petter.von.dolwitz@gmail.com> wrote= :
You can also= explore HDFS ACLs to solve this problem. This is something we use to give = different user groups different privileges. See https://hadoop.apache.org/docs<= wbr>/r2.6.0/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.= html#ACLs_Shell_Commands

Br,
Petter

<= div class=3D"gmail_quote">2017-12-06 1:12 GMT+01:00 Jim Apple <jbapple@= cloudera.com>:
I'm not sure. Some things that come to mind:

Di= d you try setting a umask for the impala user?

Did= you try setting the parent directory mode bits to 764 and then using the i= nsert_inherit_permissions flag?

On Tue, Dec 5, 2017 at 1:27 PM, Sunil= Parmar <sunilosunil@gmail.com> wrote:
Environment CDH 5.12

Wh= en running INSERT query on the table all files are always owned by user imp= ala i.e. 744, all for impala read for everyone else . We have externally ru= nning compaction process which needs to read/write/replace this files. Is t= here a way to change this default behavior to have the file permission diff= erent than 744 ? I'd prefer if it's 764 ( group read/write ) so we = can add the user to the same group as Impala who run the compaction process= .
=

Sunil Parmar





--001a1144683042973b055fb06b74--