hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kannan Muthukkaruppan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6980) Parallel Flushing Of Memstores
Date Thu, 18 Oct 2012 03:10:03 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478622#comment-13478622
] 

Kannan Muthukkaruppan commented on HBASE-6980:
----------------------------------------------

Todd: If RS zk expires, and master initiates recovery/log splitting, then the first step is
to rename the log directory from .logs/rs to .logs/rs-splitting. And then the lease recovery
is done on the individual files within the directory. Because of the directory name, any attempt
by the old RS to delete any old log files (in the old path) should fail. Therefore, still
not seeing the value of writing the flush marker.
                
> Parallel Flushing Of Memstores
> ------------------------------
>
>                 Key: HBASE-6980
>                 URL: https://issues.apache.org/jira/browse/HBASE-6980
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Kannan Muthukkaruppan
>            Assignee: Kannan Muthukkaruppan
>
> For write dominated workloads, single threaded memstore flushing is an unnecessary bottleneck.
With a single flusher thread, we are basically not setup to take advantage of the aggregate
throughput that multi-disk nodes provide.
> * For puts with WAL enabled, the bottleneck is more likely the "single" WAL per region
server. So this particular fix may not buy as much unless we unlock that bottleneck with multiple
commit logs per region server. (Topic for a separate JIRA-- HBASE-6981).
> * But for puts with WAL disabled (e.g., when using HBASE-5783 style fast bulk imports),
we should be able to support much better ingest rates with parallel flushing of memstores.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message