hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9521) clean clearBufferOnFail behavior and deprecate it
Date Tue, 17 Sep 2013 00:51:52 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769037#comment-13769037
] 

Hudson commented on HBASE-9521:
-------------------------------

FAILURE: Integrated in hbase-0.96-hadoop2 #31 (See [https://builds.apache.org/job/hbase-0.96-hadoop2/31/])
HBASE-9521  clean clearBufferOnFail behavior and deprecate it (nkeywal: rev 1523786)
* /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
* /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTableInterface.java
* /hbase/branches/0.96/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
* /hbase/branches/0.96/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncProcess.java
* /hbase/branches/0.96/hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedList.java
* /hbase/branches/0.96/hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestLoadAndVerify.java
* /hbase/branches/0.96/hbase-it/src/test/java/org/apache/hadoop/hbase/trace/IntegrationTestSendTraceRequests.java
* /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
* /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java
* /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/MultiTableOutputFormat.java
* /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java
* /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
* /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
* /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
* /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHTableUtil.java
* /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
* /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java
* /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java
* /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java
* /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
* /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationChangingPeerRegionservers.java
* /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSmallTests.java
* /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java

                
> clean clearBufferOnFail behavior and deprecate it
> -------------------------------------------------
>
>                 Key: HBASE-9521
>                 URL: https://issues.apache.org/jira/browse/HBASE-9521
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.98.0, 0.96.0
>            Reporter: Nicolas Liochon
>            Assignee: Nicolas Liochon
>            Priority: Critical
>             Fix For: 0.98.0, 0.96.0
>
>         Attachments: 9521.v1.patch, 9521.v1.patch, 9521.v2.patch, 9521.v3.patch, 9521.v3.patch
>
>
> The behavior with clearBufferOnFail is very fishy.
> {code}
>  /**
>    * When you turn {@link #autoFlush} off, you should also consider the
>    * {@link #clearBufferOnFail} option. By default, asynchronous {@link Put}
>    * requests will be retried on failure until successful. However, this can
>    * pollute the writeBuffer and slow down batching performance. Additionally,
>    * you may want to issue a number of Put requests and call
>    * {@link #flushCommits()} as a barrier. In both use cases, consider setting
>    * clearBufferOnFail to true to erase the buffer after {@link #flushCommits()}
>    * has been called, regardless of success.
>    *
>    * @param autoFlush
>    *          Whether or not to enable 'auto-flush'.
>    * @param clearBufferOnFail
>    *          Whether to keep Put failures in the writeBuffer
>    * @see #flushCommits
>    */
>   public void setAutoFlush(boolean autoFlush, boolean clearBufferOnFail) {
>     this.autoFlush = autoFlush;
>     this.clearBufferOnFail = autoFlush || clearBufferOnFail;  <============ yo man
>   }
> {code}
> {code}
>   public void setAutoFlush(boolean autoFlush) {
>     setAutoFlush(autoFlush, autoFlush); <============ more yo
>   }
> {code}
> So by default, a HTable has
> - autoflush == true
> - clearBufferOnFail == true
> BUT, if you call setAutoFlush(false), you have
> - autoflush == false
> - clearBufferOnFail == false
> So:
>  - you're setting two parameters instead of only one, without being told so.
>  - a side effect is that failed operations will be tried twice:
>        - one in the standard process
>        - one in the table close, as we're flushing the buffer again
> I would like to:
>  - deprecate clearBufferOnFail.
>  - deprecate setAutoFlush(boolean), to make things clear about what we're doing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message