hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Liochon (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-9521) clean clearBufferOnFail behavior and deprecate it
Date Thu, 12 Sep 2013 09:25:52 GMT
Nicolas Liochon created HBASE-9521:

             Summary: clean clearBufferOnFail behavior and deprecate it
                 Key: HBASE-9521
                 URL: https://issues.apache.org/jira/browse/HBASE-9521
             Project: HBase
          Issue Type: Bug
          Components: Client
    Affects Versions: 0.98.0, 0.96.0
            Reporter: Nicolas Liochon
            Assignee: Nicolas Liochon
            Priority: Critical
             Fix For: 0.98.0, 0.96.0

The behavior with clearBufferOnFail is very fishy.

   * When you turn {@link #autoFlush} off, you should also consider the
   * {@link #clearBufferOnFail} option. By default, asynchronous {@link Put}
   * requests will be retried on failure until successful. However, this can
   * pollute the writeBuffer and slow down batching performance. Additionally,
   * you may want to issue a number of Put requests and call
   * {@link #flushCommits()} as a barrier. In both use cases, consider setting
   * clearBufferOnFail to true to erase the buffer after {@link #flushCommits()}
   * has been called, regardless of success.
   * @param autoFlush
   *          Whether or not to enable 'auto-flush'.
   * @param clearBufferOnFail
   *          Whether to keep Put failures in the writeBuffer
   * @see #flushCommits
  public void setAutoFlush(boolean autoFlush, boolean clearBufferOnFail) {
    this.autoFlush = autoFlush;
    this.clearBufferOnFail = autoFlush || clearBufferOnFail;  <============ yo man

  public void setAutoFlush(boolean autoFlush) {
    setAutoFlush(autoFlush, autoFlush); <============ more yo

So by default, a HTable has
- autoflush == true
- clearBufferOnFail == true

BUT, if you call setAutoFlush(false), you have
- autoflush == false
- clearBufferOnFail == false

 - you're setting two parameters instead of only one.
 - the javadoc about writeBuffer beeing involved in the retry process was may be true 5 years
ago, but now it's wrong

I'm pretty sure that most people don't know this. You need to go to the implementation to
learn it, as the javadoc says the opposite.

a side effect is that failed operations will be tried twice:
 - one in the standard process
 - one in the table close, as we're flushing the buffer again

I would like to:
 - deprecate clearBufferOnFail.
 - change setAutoFlush(boolean) to make it change only the autoFlush, w/o activating clearBufferOnFail

It won't change the interface; but would change the behavior. I would like to put this into
the next 0.96 release.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message