Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 1 Mar 2017 22:33:46 +0000 (UTC)
From: "Ted Yu (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.13047161.1488335111000.36598.1488407626084@Atlassian.JIRA>
In-Reply-To: <JIRA.13047161.1488335111000@Atlassian.JIRA>
References: <JIRA.13047161.1488335111000@Atlassian.JIRA> <JIRA.13047161.1488335111851@jira-lw-us.apache.org>
Subject: [jira] [Updated] (HBASE-17710) HBase in standalone mode creates
 directories with 777 permission
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 01 Mar 2017 22:33:54 -0000


     [ https://issues.apache.org/jira/browse/HBASE-17710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ted Yu updated HBASE-17710:
---------------------------
    Attachment: 17710.v4.txt

bq. caveat of running in this standalone mode

I think so.

Patch v4 adds comment which refers to this JIRA.

> HBase in standalone mode creates directories with 777 permission
> ----------------------------------------------------------------
>
>                 Key: HBASE-17710
>                 URL: https://issues.apache.org/jira/browse/HBASE-17710
>             Project: HBase
>          Issue Type: Bug
>          Components: regionserver
>    Affects Versions: 1.1.2
>         Environment: HDP-2.5.3
>            Reporter: Toshihiro Suzuki
>            Assignee: Ted Yu
>         Attachments: 17710.branch-1.v1.txt, 17710.v1.txt, 17710.v2.txt, 17710.v3.txt, 17710.v4.txt
>
>
> HBase in standalone mode creates directories with 777 permission in hbase.rootdir.
> Ambari metrics collector defaults to standalone mode.
> {code}
> # find /var/lib/ambari-metrics-collector/hbase -perm 777 -type d -exec ls -ld {} \;
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/hbase/namespace/d0cca53847904f4b4add1caa0ce3a9af/info
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/hbase/backup/cbceb8fccd968b4b4583365d4dc6e377/meta
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/hbase/backup/cbceb8fccd968b4b4583365d4dc6e377/session
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/SYSTEM.CATALOG/2f4ce2294cd21cecb58fd1aca5646144/0
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/SYSTEM.SEQUENCE/0eb67274ece8a4a26cfeeef2c6d4cd37/0
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/SYSTEM.SEQUENCE/aef86710a4005f98e2dc90675f2eb325/0
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/SYSTEM.STATS/5b1d955e255e55979621214a7e4083b8/0
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/SYSTEM.FUNCTION/32c033735cf144bac5637de23f7f7dd0/0
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/METRICS_METADATA/e420dfa799742fe4516ad1e4deefb793/0
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/HOSTED_APPS_METADATA/110be63e2a9994121fc5b48d663daf2c/0
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/CONTAINER_METRICS/a103719f87e8430635abf51a7fe98637/0
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/cdb1d032beb90e350ce309e5d383c78e/0
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD/294deab47187494e845a5199702b4d04/0
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/1a263b4fe068ef2db5ba1c3e45553354/0
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_MINUTE/48f94dfb0161d8a28f645d2e1a473235/0
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_HOURLY/6d096ac3e70e54dd4a8612e17cfc4b11/0
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_RECORD_DAILY/e81850d62da64c8d1c67be309f136e23/0
> drwxrwxrwx. 2 ams hadoop 45 Mar  1 02:21 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE/b43ff796de887197834ad62fdb612b59/0
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:21 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE/b43ff796de887197834ad62fdb612b59/.tmp
> drwxrwxrwx. 2 ams hadoop 45 Mar  1 02:21 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE/c8eadeb7dead8fda9729b8e9b10c4929/0
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:21 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE/c8eadeb7dead8fda9729b8e9b10c4929/.tmp
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:17 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE_MINUTE/ca9f9754ae9ae4cdc3e1b0523eecc390/0
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:18 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE_HOURLY/8412e8a8aec5d6307943fac78ce14c7a/0
> drwxrwxrwx. 2 ams hadoop 6 Mar  1 02:18 /var/lib/ambari-metrics-collector/hbase/data/default/METRIC_AGGREGATE_DAILY/7c3358aba91ea0d76ddd8bc3ceb2d578/0
> {code}
> My analysis is as follows:
> FileSystem.mkdirs(Path f) method creates a directory with permission 777. Because HFileSystem which inherits FileSystem doesn't override the method, when we call HFileSystem.mkdirs(Path f), it tries to create a directory with permission 777.
> I've found that HFileSystem.mkdirs(Path f) gets called in the following areas:
> https://github.com/hortonworks/hbase/blob/HDP-2.5.3.0-37-tag/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java#L686
> https://github.com/hortonworks/hbase/blob/HDP-2.5.3.0-37-tag/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionFileSystem.java#L968
> At these times, the call path is as follows ("fs" is an instance of the abstract FileSystem class, being a field / member variable of FilterFileSystem):
> HFileSystem.mkdirs(Path f) -> FileSystem.mkdirs(Path f) -> FilterFileSystem.mkdirs(Path f, FsPermission permission) -> fs.mkdirs(Path f, FsPermission permission)
> For example:
> * In HRegionFileSystem.createDir(Path dir), fs.mkdirs(dir) is called. 
> * Here "fs" is an instance of HFileSystem.
> * HFileSystem who inherits from FileSystem does not have an overridden implementation of mkdirs(dir).
> * Therefore, the parent FileSystem.mkdirs(Path f) is called.
> * FileSystem.mkdirs(Path f) calls FileSystem.mkdirs(f, FsPermission.getDirDefault()).
> ** FsPermission.getDirDefault() returns a default permission of 777 via this line:
> *** return new FsPermission((short)00777);
> * FileSystem.mkdirs(f, FsPermission.getDirDefault()) is abstract, so FilterFileSystem.mkdirs(Path f, FsPermission permission) gets called.
> * FilterFileSystem.mkdirs(Path f, FsPermission permission) calls fs.mkdirs(f, permission).
> * "fs" at this point can be either RawLocalFileSystem or DistributedFileSystem depending on whether we are in standalone mode or distributed mode.
> RawLocalFileSystem.mkdirs creates directories with permission 777. On the other hand, DistributedFileSystem.mkdirs applies the umask to the 777 permission (https://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html#Changes_to_the_File_System_API). By default, the umask (fs.permissions.umask-mode) is 0022, so the permission of the directory becomes 755 after application of the 0022 umask.
> In distributed mode directories with 777 are not created in HDFS, but in standalone mode directories with 777 are created.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)