hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manoj Govindassamy (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-12652) INodeAttributesProvider#getAttributes(): Avoid multiple conversions of path components byte[][] to String[] when requesting INode attributes
Date Thu, 12 Oct 2017 23:24:00 GMT
Manoj Govindassamy created HDFS-12652:
-----------------------------------------

             Summary: INodeAttributesProvider#getAttributes(): Avoid multiple conversions
of path components byte[][] to String[] when requesting INode attributes
                 Key: HDFS-12652
                 URL: https://issues.apache.org/jira/browse/HDFS-12652
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: hdfs
    Affects Versions: 3.0.0-beta1
            Reporter: Manoj Govindassamy
            Assignee: Manoj Govindassamy


{{INodeAttributesProvider#getAttributes}} needs the path components passed in to be an array
of Strings. Where as the INode and related layers maintain path components as an array of
byte[]. So, these layers are required to convert each byte[] component of the path back into
a string and for multiple times when requesting for INode attributes from the Provider. 

That is, the path "/a/b/c" requires calling the attribute provider with: (1) "", (2) "", "a",
(3) "", "a","b", (4) "", "a","b", "c". Every single one of those strings were freshly (re)converted
from a byte[]. Say, a file listing is done on a huge directory containing 100s of millions
of files, then these multiple time redundant conversions of byte[][] to String[] create lots
of tiny object garbages, occupying memory and affecting performance. Better if we could avoid
creating redundant copies of path component strings.
  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message