accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ben Popp (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-884) Take advantage of short circuit read for local files
Date Wed, 10 Jul 2013 15:59:48 GMT


Ben Popp commented on ACCUMULO-884:

I was testing Accumulo 1.5 on Hadoop 1.1.2 in order to evaluate the impact of using Fusion
IO solid-state drives to accelerate Accumulo random read TP, and I have some anecdotal results.

I added the following properties to my config:



Accumulo was able to startup with the new properties, though I did see a lot of the following
warnings from various processes
[conf.ConfigSanityCheck] WARN : BAD CONFIG unrecognized property key (

In my experiments on these solid state drives, enabling short-circuit reads more than doubled
my read throughput! (TP measured in ops/s in a YCSB-derived read-only workload test.)    
> Take advantage of short circuit read for local files
> ----------------------------------------------------
>                 Key: ACCUMULO-884
>                 URL:
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: docs
>            Reporter: Billie Rinaldi
>            Assignee: Keith Turner
> This is a new feature in hadoop 1.0.x and some versions of 0.22 and 0.23.  It allows
a client to read directly from disk instead of through a DataNode when the data is stored
locally.  Enabling it involves setting two configuration parameters, the first in hdfs-site.xml
and the second in accumulo-site.xml.  We should make sure this works with Accumulo and recommend
it in the documentation.
> - dfs.block.local-path-access.user is the key in datanode configuration to specify the
user allowed to do short circuit read.
> - is the key to enable short circuit read at the client
side configuration.
> See HDFS-2246 and for more information.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message