Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 31 Mar 2017 20:39:41 +0000 (UTC)
From: "Yi Liang (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.13060786.1490991486000.179747.1490992781709@Atlassian.JIRA>
In-Reply-To: <JIRA.13060786.1490991486000@Atlassian.JIRA>
References: <JIRA.13060786.1490991486000@Atlassian.JIRA> <JIRA.13060786.1490991486553@jira-lw-us.apache.org>
Subject: [jira] [Updated] (HBASE-17861) Regionserver down when checking the
 permission of staging dir if hbase.rootdir is on S3
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 31 Mar 2017 20:39:46 -0000


     [ https://issues.apache.org/jira/browse/HBASE-17861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yi Liang updated HBASE-17861:
-----------------------------
    Description: 
Found some issue, when set up HBASE-17437: Support specifying a WAL directory outside of the root directory.

The region server are  showdown when I add following config into hbase-site.xml 
hbase.rootdir = hdfs://xx/xx
hbase.wal.dir = s3a://xx//xx
hbase.coprocessor.region.classes = org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
Error is below
{noformat}
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw java.lang.IllegalStateException: Directory already exists but permissions aren't set to '-rwx--x--x'
{noformat}

The reason is that, when hbase enable securebulkload, hbase will create a folder in s3, it can not set above permission, because in s3, all files are listed as having full read/write permissions and all directories appear to have full rwx permissions. See Object stores have differerent authorization models in https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html


  was:
Found some issue, when set up HBASE-17437: Support specifying a WAL directory outside of the root directory.

The region server are  showdown when I add following config into hbase-site.xml 
hbase.rootdir = hdfs://xx/xx
hbase.wal.dir = s3a://xx//xx
hbase.coprocessor.region.classes = org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
Error is below
{noformat}
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw java.lang.IllegalStateException: Directory already exists but permissions aren't set to '-rwx--x--x'
{noformat}

The reason is that, when hbase enable securebulkload, hbase will create a folder in s3, it can not set above permission, because in s3, all files are listed as having full read/write permissions and all directories appear to have full rwx permissions.  See simulated permission section in https://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.8.0/bk_hdcloud-aws/content/s3-s3aclient/index.html


> Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3
> ---------------------------------------------------------------------------------------
>
>                 Key: HBASE-17861
>                 URL: https://issues.apache.org/jira/browse/HBASE-17861
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 1.4.0
>            Reporter: Yi Liang
>            Assignee: Yi Liang
>              Labels: filesystem, s3, wal
>             Fix For: 1.4.0
>
>         Attachments: HBASE-17861-V1.patch
>
>
> Found some issue, when set up HBASE-17437: Support specifying a WAL directory outside of the root directory.
> The region server are  showdown when I add following config into hbase-site.xml 
> hbase.rootdir = hdfs://xx/xx
> hbase.wal.dir = s3a://xx//xx
> hbase.coprocessor.region.classes = org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
> Error is below
> {noformat}
> org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw java.lang.IllegalStateException: Directory already exists but permissions aren't set to '-rwx--x--x'
> {noformat}
> The reason is that, when hbase enable securebulkload, hbase will create a folder in s3, it can not set above permission, because in s3, all files are listed as having full read/write permissions and all directories appear to have full rwx permissions. See Object stores have differerent authorization models in https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)