hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-894) dfs client protocol should allow asking for parts of the block map
Date Thu, 03 May 2007 20:39:15 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12493508
] 

Doug Cutting commented on HADOOP-894:
-------------------------------------

I think it's strange to put LocatedBlocks in DFSFileInfo.  You're trying to optimize the protocol,
so that a separate call isn't required to get the length, right?  So let's make that explicit
by returning the file length along with the list of blocks, rather than hacking DFSFileInfo.

public LocatedBlocks {
  private LocatedBlock[] blocks;
  private long fileLength;
}

public LocatedBlocks getBlockLocations(String file, long start, long length);

Then we don't need the open() method at all.  getBlockLocations() replaces it altogether.
 This also has the benefit that someone can open a file in the middle with a single RPC.


> dfs client protocol should allow asking for parts of the block map
> ------------------------------------------------------------------
>
>                 Key: HADOOP-894
>                 URL: https://issues.apache.org/jira/browse/HADOOP-894
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Owen O'Malley
>         Assigned To: Konstantin Shvachko
>             Fix For: 0.13.0
>
>         Attachments: partialBlockList.patch, partialBlockList2.patch
>
>
> I think that the HDFS client protocol should change like:
> /** The meta-data about a file that was opened. */
> class OpenFileInfo {
>   /** the info for the first block */
>   public LocatedBlockInfo getBlockInfo();
>   public long getBlockSize();
>   public long getLength();
> }
> interface ClientProtocol extends VersionedProtocol {
>   public OpenFileInfo open(String name) throws IOException;
>   /** get block info for any range of blocks */
>   public LocatedBlockInfo[] getBlockInfo(String name, int blockOffset, int blockLength)
throws IOException;
> }
> so that the client can decide how much block info to request and when. Currently, when
the file is opened or an error occurs, the entire block list is requested and sent.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message