hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HDFS-4697) short-circuit reads do not honor readahead settings
Date Wed, 01 Oct 2014 18:37:35 GMT

     [ https://issues.apache.org/jira/browse/HDFS-4697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Colin Patrick McCabe resolved HDFS-4697.
----------------------------------------
       Resolution: Fixed
    Fix Version/s: 2.3.0

Short-circuit now honors {{dfs.datanode.readahead.bytes}}, since HDFS-5634.

However, if you want short-circuit the full 4 MB of readahead that the datanode is doing,
you'll need to set {{dfs.client.read.shortcircuit.buffer.size}} to 4 megabytes in your configuration.
 Its value is specified in bytes and it defaults to 1 megabyte.  The tradeoff is slightly
increased memory consumption.

> short-circuit reads do not honor readahead settings
> ---------------------------------------------------
>
>                 Key: HDFS-4697
>                 URL: https://issues.apache.org/jira/browse/HDFS-4697
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>    Affects Versions: 2.0.3-alpha
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Minor
>             Fix For: 2.3.0
>
>
> Neither the new nor the legacy short-circuit read implementations honor {{dfs.datanode.readahead.bytes}}.
 This can result in scenarios where non-short-circuit reads are faster for long reads, simply
because they are doing more readahead, and SCR is not.  We should do readahead in both cases
if it is configured.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message