hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3504) HLog performance improvement
Date Fri, 04 Feb 2011 18:44:32 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990672#comment-12990672

stack commented on HBASE-3504:

It looks like we write them out to the recovered.edits file in the order in which we encounter
them: http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.html#659.
 RegionEntryBuffer does not sort them, http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.html#581.

Here is where we replay the recovered edits on region open: http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/HRegion.html#1837
 It looks like this code, on cursory review, is immune to out of order edits (Edits will be
sorted on insertion into memstore).

Out of order edits though could be a problem in case where we got a new value and a delete
of that same value coming in at around the same time. Out of order could change result seen
on other side of the log split.

Sorting at split time would be an option.  Would be big change.

> HLog performance improvement
> ----------------------------
>                 Key: HBASE-3504
>                 URL: https://issues.apache.org/jira/browse/HBASE-3504
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
> The HLog.updateLock protects the rolling of logs with concurrent writes to the HDFS log
file. This is a scalability bottleneck for a workload that comprises mostly of counter-increments.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message