hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2283) row level atomicity
Date Fri, 12 Mar 2010 20:51:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12844672#action_12844672

stack commented on HBASE-2283:

Here's some comments.

I think KeyValueList not the best name for a List that is purposed to appending to a WAL log.
 KVL implies List<KeyValue> which it replaces in your code but in fact it does much
more.  For one its a Writable.  It also takes care of your fancy new transaction start/end
markings.  Rename it WALEdit or HLogEdit or something?  Should it also be moved to the wal
package? (o.a.h.h.regionserver.wal).

Minor: Should it implement List<KeyValue>?  Would that help?  You wouldn't need to have
a getList?

I love the class comment on KVL.

readNonLengthData should instead be an overloading of readFields, just add the length param?

Minor: The following if (kvlist.size() > 0) { is usually cheaper if done as !kvlist.isEmpty
IIRC, especially if the list one of the concurrent implementations where size calc is expensive
(probably not pertinent here).

Its interesting, our number-of-entries in hlog will change now, now you group up puts, etc.

Patch looks great Kannan. 

> row level atomicity 
> --------------------
>                 Key: HBASE-2283
>                 URL: https://issues.apache.org/jira/browse/HBASE-2283
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: Kannan Muthukkaruppan
>            Priority: Blocker
>             Fix For: 0.20.4, 0.21.0
>         Attachments: rowLevelAtomicity_2283_v1.patch
> The flow during a HRegionServer.put() seems to be the following. [For now, let's just
consider single row Put containing edits to multiple column families/columns.]
> HRegionServer.put() does a:
>         HRegion.put();
>        syncWal()  (the HDFS sync call).  /* this is assuming we have HDFS-200 */
> HRegion.put() does a:
>   for each column family 
>   {
>       HLog.append(all edits to the colum family);
>       write all edits to Memstore;
>   }
> HLog.append() does a :
>   foreach edit in a single column family {
>     doWrite()
>   }
> doWrite() does a:
>    this.writer.append().
> There seems to be two related issues here that could result in inconsistencies.
> Issue #1: A put() does a bunch of HLog.append() calls. These in turn do a bunch of "write"
calls on the underlying DFS stream.  If we crash after having written out some append's to
DFS, recovery will run and apply a partial transaction to memstore.  
> Issue #2: The updates to memstore  should happen after the sync rather than before. Otherwise,
there is the danger that the write to DFS (sync) fails for some reason & we return an
error to the client, but we have already taken edits to the memstore. So subsequent reads
will serve uncommitted data.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message