hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13321) Deprecate FileSystem APIs that promote inefficient call patterns.
Date Sat, 25 Jun 2016 00:06:16 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348923#comment-15348923
] 

Chris Nauroth commented on HADOOP-13321:
----------------------------------------

HIVE-10223 is an example of an inefficient call pattern: multiple calls to the convenience
wrapper methods that I was able to trivially consolidate down to a single {{getFileStatus}}.

I believe the full set of these wrapper methods to deprecate are:

{{FileSystem#exists}}
{{FileSystem#getBlockSize}}
{{FileSystem#getLength}}
{{FileSystem#getReplication}}
{{FileSystem#isDirectory}}
{{FileSystem#isFile}}

This issue does not apply to {{FileContext}}, where methods like {{exists}} can be placed
behind {{FileContext.Util}}, which comes with JavaDocs describing that these are not primitive
operations.

> Deprecate FileSystem APIs that promote inefficient call patterns.
> -----------------------------------------------------------------
>
>                 Key: HADOOP-13321
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13321
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs
>            Reporter: Chris Nauroth
>
> {{FileSystem}} contains several methods that act as convenience wrappers over calling
{{getFileStatus}} and retrieving a single property of the returned {{FileStatus}}.  These
methods have a habit of fostering inefficient call patterns in applications, resulting in
multiple redundant {{getFileStatus}} calls.  For HDFS, this translates into wasteful NameNode
RPC traffic.  For file systems backed by cloud object stores, this translates into wasteful
HTTP traffic.  This issue proposes to deprecate these methods and instead encourage applications
to call {{getFileStatus}} and then reuse the same {{FileStatus}} instance as needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message