hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HDFS-4697) short-circuit reads do not honor readahead settings
Date Wed, 01 Oct 2014 18:37:35 GMT

     [ https://issues.apache.org/jira/browse/HDFS-4697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Colin Patrick McCabe resolved HDFS-4697.
       Resolution: Fixed
    Fix Version/s: 2.3.0

Short-circuit now honors {{dfs.datanode.readahead.bytes}}, since HDFS-5634.

However, if you want short-circuit the full 4 MB of readahead that the datanode is doing,
you'll need to set {{dfs.client.read.shortcircuit.buffer.size}} to 4 megabytes in your configuration.
 Its value is specified in bytes and it defaults to 1 megabyte.  The tradeoff is slightly
increased memory consumption.

> short-circuit reads do not honor readahead settings
> ---------------------------------------------------
>                 Key: HDFS-4697
>                 URL: https://issues.apache.org/jira/browse/HDFS-4697
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>    Affects Versions: 2.0.3-alpha
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Minor
>             Fix For: 2.3.0
> Neither the new nor the legacy short-circuit read implementations honor {{dfs.datanode.readahead.bytes}}.
 This can result in scenarios where non-short-circuit reads are faster for long reads, simply
because they are doing more readahead, and SCR is not.  We should do readahead in both cases
if it is configured.

This message was sent by Atlassian JIRA

View raw message