hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lokesh Jain (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDDS-1834) parent directories not found in secure setup due to ACL check
Date Thu, 25 Jul 2019 14:15:00 GMT

    [ https://issues.apache.org/jira/browse/HDDS-1834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892790#comment-16892790
] 

Lokesh Jain edited comment on HDDS-1834 at 7/25/19 2:14 PM:
------------------------------------------------------------

There are two bugs associated with checkAccess.
 # In OzoneFileSystem use cases, for access of a descendant checkAccess of any ancestor
is not done. Currently while accessing a/b/c.txt we do not check the access for a/ and a/b/
and do a access check only for the path a/b/c.txt
 # In HDDS-1481 while doing mkdir, the ancestor directories are not created if they do not
exist. checkAccess method only checks for the key provided and therefore fails with KEY_NOT_FOUND
error. It should do a check for existence of a directory using getFileStatus.

KeyManagerImpl#checkAccess:1645-1657
{code:java}
OmKeyInfo keyInfo = metadataManager.getKeyTable().get(objectKey);
if (keyInfo == null) {
  objectKey = OzoneFSUtils.addTrailingSlashIfNeeded(objectKey);
  keyInfo = metadataManager.getKeyTable().get(objectKey);
  
  if(keyInfo == null) {
    keyInfo = metadataManager.getOpenKeyTable().get(objectKey);
    if (keyInfo == null) {
      throw new OMException("Key not found, checkAccess failed. Key:" +
          objectKey, KEY_NOT_FOUND);
    }
  }
}
{code}
Example illustrating the problem 2. 
{code:java}
ozone sh key list o3://om/fstest/bucket1/
[ {
"version" : 0,
"md5hash" : null,
"createdOn" : "Thu, 25 Jul 2019 11:26:02 GMT",
"modifiedOn" : "Thu, 25 Jul 2019 11:26:02 GMT",
"size" : 0,
"keyName" : "testdir/deep/",
"type" : null
}, {
"version" : 0,
"md5hash" : null,
"createdOn" : "Thu, 25 Jul 2019 11:26:09 GMT",
"modifiedOn" : "Thu, 01 Jan 1970 00:12:54 GMT",
"size" : 22808,
"keyName" : "testdir/deep/MOVED.TXT",
"type" : null
}, {
"version" : 0,
"md5hash" : null,
"createdOn" : "Thu, 25 Jul 2019 11:26:18 GMT",
"modifiedOn" : "Thu, 01 Jan 1970 00:12:44 GMT",
"size" : 22808,
"keyName" : "testdir/deep/PUTFILE.txt",
"type" : null
} ]

ozone sh key info o3://om/fstest/bucket1/testdir
KEY_NOT_FOUND Key not found, checkAccess failed. Key:/fstest/bucket1/testdir/
{code}


was (Author: ljain):
The problem exists in general for checkAccess. There are two bugs associated with checkAccess.
 # In OzoneFileSystem use cases, for access of a descendant checkAccess of any ancestor
is not done. Currently while accessing a/b/c.txt we do not check the access for a/ and a/b/
and do a access check only for the path a/b/c.txt
 # In HDDS-1481 while doing mkdir, the ancestor directories are not created if they do not
exist. checkAccess method only checks for the key provided and therefore fails with KEY_NOT_FOUND
error. It should do a check for existence of a directory using getFileStatus.

KeyManagerImpl#checkAccess:1645-1657
{code:java}
OmKeyInfo keyInfo = metadataManager.getKeyTable().get(objectKey);
if (keyInfo == null) {
  objectKey = OzoneFSUtils.addTrailingSlashIfNeeded(objectKey);
  keyInfo = metadataManager.getKeyTable().get(objectKey);
  
  if(keyInfo == null) {
    keyInfo = metadataManager.getOpenKeyTable().get(objectKey);
    if (keyInfo == null) {
      throw new OMException("Key not found, checkAccess failed. Key:" +
          objectKey, KEY_NOT_FOUND);
    }
  }
}
{code}
Example illustrating the problem 2. 
{code:java}
ozone sh key list o3://om/fstest/bucket1/
[ {
"version" : 0,
"md5hash" : null,
"createdOn" : "Thu, 25 Jul 2019 11:26:02 GMT",
"modifiedOn" : "Thu, 25 Jul 2019 11:26:02 GMT",
"size" : 0,
"keyName" : "testdir/deep/",
"type" : null
}, {
"version" : 0,
"md5hash" : null,
"createdOn" : "Thu, 25 Jul 2019 11:26:09 GMT",
"modifiedOn" : "Thu, 01 Jan 1970 00:12:54 GMT",
"size" : 22808,
"keyName" : "testdir/deep/MOVED.TXT",
"type" : null
}, {
"version" : 0,
"md5hash" : null,
"createdOn" : "Thu, 25 Jul 2019 11:26:18 GMT",
"modifiedOn" : "Thu, 01 Jan 1970 00:12:44 GMT",
"size" : 22808,
"keyName" : "testdir/deep/PUTFILE.txt",
"type" : null
} ]

ozone sh key info o3://om/fstest/bucket1/testdir
KEY_NOT_FOUND Key not found, checkAccess failed. Key:/fstest/bucket1/testdir/
{code}

> parent directories not found in secure setup due to ACL check
> -------------------------------------------------------------
>
>                 Key: HDDS-1834
>                 URL: https://issues.apache.org/jira/browse/HDDS-1834
>             Project: Hadoop Distributed Data Store
>          Issue Type: Bug
>          Components: Ozone Filesystem
>            Reporter: Doroszlai, Attila
>            Assignee: Doroszlai, Attila
>            Priority: Blocker
>
> ozonesecure-ozonefs acceptance test is failing, because {{ozone fs -mkdir -p}} only creates
key for the specific directory, not its parents.
> {noformat}
> ozone fs -mkdir -p o3fs://bucket1.fstest/testdir/deep
> {noformat}
> Previous result:
> {noformat:title=https://ci.anzix.net/job/ozone-nightly/176/artifact/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/compose/result/log.html#s1-s16-t3-k2}
> $ ozone sh key list o3://om/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
> testdir/
> testdir/deep/
> {noformat}
> Current result:
> {noformat:title=https://ci.anzix.net/job/ozone-nightly/177/artifact/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/compose/result/log.html#s1-s16-t3-k2}
> $ ozone sh key list o3://om/fstest/bucket1 | grep -v WARN | jq -r '.[].keyName'
> testdir/deep/
> {noformat}
> The failure happens on first operation that tries to use {{testdir/}} directly:
> {noformat}
> $ ozone fs -touch o3fs://bucket1.fstest/testdir/TOUCHFILE.txt
> ls: `o3fs://bucket1.fstest/testdir': No such file or directory
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message