hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2246) Shortcut a local client reads to a Datanodes files directly
Date Sat, 05 Nov 2011 02:17:51 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144522#comment-13144522

Tsz Wo (Nicholas), SZE commented on HDFS-2246:

- For DataNode.getBlockLocalPathInfo(..),
-* DataNode should check DFS_CLIENT_READ_SHORTCIRCUIT.
-* Should it calls checkBlockToken?
-* Should it checks whether the client address is local?

- DataNode.getBlockLocalPathInfo(..) is not supposed to calls FSDataset.getMetaFile(..) directly.
 It should only calls FSDatasetInterface methods.  How about change the new FSDatasetInterface.getBlockFile(..)
to getBlockLocalPathInfo(..) wich returns BlockLocalPathInfo?

- Change DataNode.userWithLocalPathAccess to final.

- SoftLRUCache is implemented with SoftReference values.  In general, it is a useful data
structure.  However, it may not be useful here since the size of the cache is about 10k and
the size of value is small (< 1k).  I suggest using a synchronized LinkedHashMap (no SoftReference)
in BlockReaderLocal.cache and remove SoftLRUCache from the patch.

- BlockReaderLocal.cache should not be initialized if checkShortCircuit is not enabled.

I have not yet finished reading the client code.

> Shortcut a local client reads to a Datanodes files directly
> -----------------------------------------------------------
>                 Key: HDFS-2246
>                 URL: https://issues.apache.org/jira/browse/HDFS-2246
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Sanjay Radia
>         Attachments: 0001-HDFS-347.-Local-reads.patch, HDFS-2246-branch-0.20-security-205.patch,
HDFS-2246-branch-0.20-security-205.patch, HDFS-2246-branch-0.20-security.patch, HDFS-2246.20s.1.patch,
HDFS-2246.20s.2.txt, HDFS-2246.20s.3.txt, HDFS-2246.20s.4.txt, HDFS-2246.20s.patch, localReadShortcut20-security.2patch

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message