hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2465) Add HDFS support for fadvise readahead and drop-behind
Date Thu, 27 Oct 2011 18:34:32 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13137374#comment-13137374
] 

Eli Collins commented on HDFS-2465:
-----------------------------------

Looked at the patch, think this is a reasonable first DN-wide heuristic. Comments follow:

* There isn't any config that enables syncBehindWrites in BlockerRecevier.
* Would it be more clear to express LONG_READ_THRESHOLD in terms of a multiple of the packet
size or the IO file buffer size? Seems like the host readahead should match the implicit HDFS
readahead.
* In BlockerSender#sendBlock by the 1st fadvise a comment like "advice applies to the entire
file" would help
* Nit: add _BYTES to LONG_READ_THRESHOLD and CACHE_DROP_INTERVAL. 

Otherwise looks great.
                
> Add HDFS support for fadvise readahead and drop-behind
> ------------------------------------------------------
>
>                 Key: HDFS-2465
>                 URL: https://issues.apache.org/jira/browse/HDFS-2465
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: data-node
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hdfs-2465.txt
>
>
> This is the HDFS side of HADOOP-7714. The initial implementation is heuristic based and
should be considered experimental, as discussed in the parent JIRA. It should be off by default
until better heuristics, APIs, and tuning experience is developed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message