hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kannan Muthukkaruppan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2283) row level atomicity
Date Tue, 09 Mar 2010 18:53:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12843240#action_12843240
] 

Kannan Muthukkaruppan commented on HBASE-2283:
----------------------------------------------


Have coded the changes to make all the "appends" happen first, then "sync" and then all the
memstore edits are done. 

Now, trying to work on collapsing all the appends into a single append operation. The marker
can simply be the number of edits we expect to find in the append operation. 

Today, if a transaction contains 3 edits to c1, c2, c3 for row R. The HLog contains, has three
log entries:

<logseq1-for-edit1>:<KeyValue-for-edit-c1>
<logseq2-for-edit2>:<KeyValue-for-edit-c2>
<logseq3-for-edit3>:<KeyValue-for-edit-c3>

In the new model, the plan is to have something like:

<logseq#-for-entire-txn>:<3, <Keyvalue-for-edit-c1>, <KeyValue-for-edit-c2>,
<KeyValue-for-edit-c3>>

Implementing the change itself should be reasonably straightforward. 

But one quick question: In the new version, I presumably have to  worry about supporting both
the old & the new log format, correct? (i.e. for cases when someone upgraded from an older
version of HBase to this version)





> row level atomicity 
> --------------------
>
>                 Key: HBASE-2283
>                 URL: https://issues.apache.org/jira/browse/HBASE-2283
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: Kannan Muthukkaruppan
>            Priority: Blocker
>             Fix For: 0.20.4, 0.21.0
>
>
> The flow during a HRegionServer.put() seems to be the following. [For now, let's just
consider single row Put containing edits to multiple column families/columns.]
> HRegionServer.put() does a:
>         HRegion.put();
>        syncWal()  (the HDFS sync call).  /* this is assuming we have HDFS-200 */
> HRegion.put() does a:
>   for each column family 
>   {
>       HLog.append(all edits to the colum family);
>       write all edits to Memstore;
>   }
> HLog.append() does a :
>   foreach edit in a single column family {
>     doWrite()
>   }
> doWrite() does a:
>    this.writer.append().
> There seems to be two related issues here that could result in inconsistencies.
> Issue #1: A put() does a bunch of HLog.append() calls. These in turn do a bunch of "write"
calls on the underlying DFS stream.  If we crash after having written out some append's to
DFS, recovery will run and apply a partial transaction to memstore.  
> Issue #2: The updates to memstore  should happen after the sync rather than before. Otherwise,
there is the danger that the write to DFS (sync) fails for some reason & we return an
error to the client, but we have already taken edits to the memstore. So subsequent reads
will serve uncommitted data.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message