hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4453) Make a simple doc to describe the usage and design of the shortcircuit read feature
Date Sat, 02 Feb 2013 00:52:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569328#comment-13569328
] 

Todd Lipcon commented on HDFS-4453:
-----------------------------------

Section 4: I think you have some typos there:
- you mean GetBlockLocalPathInfo, not GetBlockLocations
- you said 'clients' twice where I think one of them should read 'servers'

{code}
+  To configure short-circuit local reads, you will need to put
+  <<<libhadoop.so>>> in your <<<LD_LIBRARY_PATH>>>. 
You can check if you have done
+  this by running 
+
{code}

Users don't generally set this via {{LD_LIBRARY_PATH}}. Instead, it goes in the appropriate
directory inside the install tree. Do we have other docs already about how to enable the native
code? Might be better to refer to those.

----
- Regarding {{/var/lib/hadoop-hdfs}} vs {{/var/run/hadoop-hdfs}} -- why's it problematic if
/var/run is a tmpfs? We shouldn't need it to persist cross-reboot, and /var/run is generally
_not_ cleaned by a tmpwatch process. tmpfs is also better in that it will continue to work
even if a local disk dies.

- In the example config, I would not use the _PORT trick - it only really makes sense for
dev setups like the minicluster, and otherwise may just confuse the user.

- Please specify that you need to set these two configurations both on clients and on servers.
                
> Make a simple doc to describe the usage and design of the shortcircuit read feature
> -----------------------------------------------------------------------------------
>
>                 Key: HDFS-4453
>                 URL: https://issues.apache.org/jira/browse/HDFS-4453
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, hdfs-client
>            Reporter: Brandon Li
>            Assignee: Colin Patrick McCabe
>         Attachments: HDFS-4453.001.patch, HDFS-4453.002.patch
>
>
> It would be nice to have a document to describe the configuration and design of this
feature. Also its relationship with previous short circuit read implementation(HDFS-2246),
for example, can they co-exist, or this one is planed to replaces HDFS-2246, or it can fall
back on HDFS-2246 when unix domain socket is not supported.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message