hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4528) The put operation can release the rowlock before sync-ing the Hlog
Date Thu, 06 Oct 2011 01:22:30 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121652#comment-13121652
] 

jiraposter@reviews.apache.org commented on HBASE-4528:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2141/#review2373
-----------------------------------------------------------


Two potential issues that I see:

#1 - If I'm reading it correctly, this change breaks current behavior for tables where DEFERRED_LOG_FLUSH=true,
where HLog.sync() should not be triggered.

#2 - It also seems like moving the memstore updates prior to the WAL append and sync has the
potential to leave around memstore entries that are not present in the WAL if:
1) the log.sync() call throws an IOException
2) the memstore entries are made visible via the call to rwcc.completeMemstoreInsert() in
the finally block
3) the client sees the puts as failed due to the IOException
4) the subsequent roll of the HLog writer triggered by the IOException succeeds -- the region
server does not abort in this case.  

In this case, wouldn't the client see the update as failed (due to the IOException), while
the memstore entries persist?  So do the memstore changes need to be tracked and rolled back?
 Or am I missing something?


/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
<https://reviews.apache.org/r/2141/#comment5460>

    Doesn't this break current behavior in respect to tables with deferred log flush set to
"true"?  Previously no sync was triggered for these tables.
    
    Also, if the sync call throws an IOException that gets sent back to the client, how do
the memstore entries get cleared?


- Gary


On 2011-10-03 05:54:07, Dhruba Borthakur wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2141/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-10-03 05:54:07)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  The changes the multiPut operation so that the sync to the wal occurs outside the rowlock.
bq.  
bq.  This enhancement is done only to HRegion.mut(Put[]) because this is the only method that
gets invoked from an application. The HRegion.put(Put) is used only by unit tests and should
possibly be deprecated.
bq.  
bq.  I have attached a unit test. I have not yet run all unit tests, but early feedback on
this patch will be very helpful.
bq.  
bq.  
bq.  This addresses bug HBASE-4528.
bq.      https://issues.apache.org/jira/browse/HBASE-4528
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1178298 
bq.    /src/test/java/org/apache/hadoop/hbase/regionserver/TestParallelPut.java PRE-CREATION

bq.  
bq.  Diff: https://reviews.apache.org/r/2141/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  Not yet run the full suite of unit tests.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Dhruba
bq.  
bq.


                
> The put operation can release the rowlock before sync-ing the Hlog
> ------------------------------------------------------------------
>
>                 Key: HBASE-4528
>                 URL: https://issues.apache.org/jira/browse/HBASE-4528
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: appendNoSyncPut1.txt, appendNoSyncPut2.txt
>
>
> This allows for better throughput when there are hot rows. A single row update improves
from 100 puts/sec/server to 5000 puts/sec/server.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message