hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4056) Support for using faster storage for write-ahead log
Date Fri, 08 Jul 2011 06:29:17 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13061786#comment-13061786

Jonathan Gray commented on HBASE-4056:

Thanks for opening this JIRA.

What do you see as the primary benefit of using flash for the WAL?  I've seen some improvement
in sequential write throughput, but not drastically different.

It seems to me that a significant benefit of using flash is the fast random read access, and
there are no random reads on the WAL.

One idea that has floated around is to do something like cache-on-write to copy recently written
files onto flash (in addition to HDFS) to allow for fast random read access.  Or use flash
as some kind of extension to the block cache.

But regardless, making all of this stuff configurable and supporting more diverse setups is
a good thing in general.  Some experiments and benchmarks around this would be awesome.  Good

> Support for using faster storage for write-ahead log
> ----------------------------------------------------
>                 Key: HBASE-4056
>                 URL: https://issues.apache.org/jira/browse/HBASE-4056
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Praveen Kumar
>            Priority: Minor
>              Labels: features
> On clusters with heterogeneous storage components like hard drives and flash memory,
it could be beneficial to use flash memory for write-ahead log. This can be accomplished by
using client side mount table support (HADOOP-7257) that is offered by HDFS federation (HDFS-1052)
feature. One can define two HDFS namespaces (faster and slower), and configure HBase to use
faster storage namespace for storing WAL.
> This is an abstract task that captures the idea. More brainstorming and subtasks identification
to follow.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message