hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5304) Expose if a block replica is cached in getFileBlockLocations
Date Mon, 07 Oct 2013 21:51:42 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13788626#comment-13788626

Colin Patrick McCabe commented on HDFS-5304:

An array of booleans in {{LocatedBlock}} seems kind of clunky.  You have to check all of them
to find out if any are true.    And then you have to match them up with the corresponding
element in the {{locs}} array.  Can we have something like this instead?
DatanodeInfo[] cachedLocs;

Obviously, these would just be pointers to the same DatanodeInfo objects stored in {{locs}},
not copies.  This way:
* it's easy to iterate over all cached locations, or check if there are any (just do {{location.getCachedLocations.isEmpty()}}).
* this can be mapped to a statically declared empty array most of the time, saving memory

The RPC changes themselves are fine, I think.

> Expose if a block replica is cached in getFileBlockLocations
> ------------------------------------------------------------
>                 Key: HDFS-5304
>                 URL: https://issues.apache.org/jira/browse/HDFS-5304
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: HDFS-4949
>            Reporter: Andrew Wang
>            Assignee: Andrew Wang
>         Attachments: hdfs-5304-1.patch
> We need to expose which replicas of a block are cached so applications can place their
tasks for memory-locality.

This message was sent by Atlassian JIRA

View raw message