accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-884) Take advantage of short circuit read for local files
Date Wed, 10 Jul 2013 16:43:48 GMT


Josh Elser commented on ACCUMULO-884:

Ah ok. Found my notes.

Spinning disks and cdh3u5, I saw 10x read speedup when I enabled the hadoop native-libs with
a test based on the TestShortCircuitLocalRead from HDFS-2246. I'm curious what kind of increase
you see directly against HDFS and if that result is in line with what I saw way back when.

bq. If I remember correctly, I read somewhere that you won't see any benefit from shortcircuit
reads until you actually get to hadoop-2 (maybe 0.23 too?). I'll see if I can find that information

I can't find any more info on this, so ignore me for the time being :P
> Take advantage of short circuit read for local files
> ----------------------------------------------------
>                 Key: ACCUMULO-884
>                 URL:
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: docs
>            Reporter: Billie Rinaldi
>            Assignee: Keith Turner
> This is a new feature in hadoop 1.0.x and some versions of 0.22 and 0.23.  It allows
a client to read directly from disk instead of through a DataNode when the data is stored
locally.  Enabling it involves setting two configuration parameters, the first in hdfs-site.xml
and the second in accumulo-site.xml.  We should make sure this works with Accumulo and recommend
it in the documentation.
> - dfs.block.local-path-access.user is the key in datanode configuration to specify the
user allowed to do short circuit read.
> - is the key to enable short circuit read at the client
side configuration.
> See HDFS-2246 and for more information.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message