pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Created] (PIG-4442) Eliminate redundant RPC call to get file information in HPath.
Date Tue, 03 Mar 2015 01:19:04 GMT
Chris Nauroth created PIG-4442:
----------------------------------

             Summary: Eliminate redundant RPC call to get file information in HPath.
                 Key: PIG-4442
                 URL: https://issues.apache.org/jira/browse/PIG-4442
             Project: Pig
          Issue Type: Improvement
    Affects Versions: 0.13.0
            Reporter: Chris Nauroth
            Assignee: Chris Nauroth
            Priority: Minor


The {{HPath}} class makes 2 separate calls to {{FileSystem#getFileStatus}} to get the block
size and replication.  In the case of HDFS, this results in 2 separate but identical RPC transactions
with the NameNode.  The situation is the same for many other alternative {{FileSystem}} implementations
too.  We can get a minor latency improvement and lighten some RPC load on the remote services
by using a single call and getting the block size and replication from the same response.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message