hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manoj Govindassamy (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-12203) INodeAttributesProvider#getAttributes() support for default/passthrough mode
Date Wed, 26 Jul 2017 20:00:02 GMT
Manoj Govindassamy created HDFS-12203:
-----------------------------------------

             Summary: INodeAttributesProvider#getAttributes() support for default/passthrough
mode
                 Key: HDFS-12203
                 URL: https://issues.apache.org/jira/browse/HDFS-12203
             Project: Hadoop HDFS
          Issue Type: Improvement
            Reporter: Manoj Govindassamy
            Assignee: Manoj Govindassamy


{{INodeAttributeProvider}}
{noformat}
  ..
  public abstract INodeAttributes getAttributes(String[] pathElements, INodeAttributes inode);

  public INodeAttributes getAttributes(byte[][] components, INodeAttributes inode) {
    String[] elements = new String[components.length];
    for (int i = 0; i < elements.length; i++) {
      elements[i] = DFSUtil.bytes2String(components[i]);
    }
    return getAttributes(elements, inode);
  }

{noformat}

The abstract method {{INodeAttributesProvider#getAttributes()}} currently takes in INode path
elements and the INode. Many a times, the provider falls back to the local INode attributes.
But, there isn't a way for the caller to tell the provider to fall back to local attribute
on demand. Below is one such usecase, where the caller of getAttributes() wants to use the
local attributes instead of the one coming from the external provider.

Assume the external attributes provider takes care of providing custom ACLFeature for some
specific directories and all its descendants (like Sentry) and fallsback to the default INode
ACLFeature for all other files/directories. And lets assume that the external provider also
has its meta data properly replicated to another site. 

An operation like snapshot diff iterates over the children of given snapshot pair and generates
the diff list. In order to generate the diff list, files are compared which involves their
metadata comparisons. Once HDFS-12201 is fixed, metadata comparisons will compare the attributes
from external providers as well. Given that external attributes can be ignored as they are
taken care by other means, Snapshot diff wants the meta data comparison to ignore the features
coming from external attribute provider and fall back to the local INode features.  This on-demand
fall-back or passthrough mode for INodeAttributesProvider will help to avoid the huge performance,
or memory over consumption problems arising out of the external provider plugins.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org


Mime
View raw message