hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17361) Make HTable thread safe
Date Sat, 24 Dec 2016 13:38:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15774912#comment-15774912

Duo Zhang commented on HBASE-17361:

If we are going to keep the table implementation not thread safe then it is not a problem
to have setXXXTimeout methods which can give user the ability to change the timeout before
making a call to HBase cluster. And it is not problem that we override the default timeout
as our best practice is to get a new table instance every time.

But here we are trying to make the interface thread safe, then it is a big problem. Even if
we make the implementation thread safe, a 'set timeout and then make a call' is still not
thread safe unless the user introduces a lock in his/her code...

And IMO, thread-safe or not-thread-safe is not a dead-or-alive choice, so in the past I think
it is better to keep the old way which gives the user a consistent experience. And [~carp84]
said he has some numbers which shows that a thread safe HTable can give a better performance,
then I think we have a good reason to change the interface to thread safe.


> Make HTable thread safe
> -----------------------
>                 Key: HBASE-17361
>                 URL: https://issues.apache.org/jira/browse/HBASE-17361
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Yu Li
>            Assignee: Yu Li
>         Attachments: HBASE-17361.patch, HBASE-17361.patch
> Currently HTable is marked as NOT thread safe, and this JIRA target at improving this
to take better usage of the thread-safe BufferedMutator.
> Some findings/work done:
> If we try to do put to the same HTable instance in parallel, there'll be problem, since
now we have {{HTable#getBufferedMutator}} like
> {code}
>    BufferedMutator getBufferedMutator() throws IOException {
>      if (mutator == null) {
>       this.mutator = (BufferedMutatorImpl) connection.getBufferedMutator(
>           new BufferedMutatorParams(tableName)
>               .pool(pool)
>               .writeBufferSize(connConfiguration.getWriteBufferSize())
>               .maxKeyValueSize(connConfiguration.getMaxKeyValueSize())
>       );
>     }
>     mutator.setRpcTimeout(writeRpcTimeout);
>     mutator.setOperationTimeout(operationTimeout);
>     return mutator;
>   }
> {code}
> And {{HTable#flushCommits}}:
> {code}
>   void flushCommits() throws IOException {
>     if (mutator == null) {
>       // nothing to flush if there's no mutator; don't bother creating one.
>       return;
>     }
>     getBufferedMutator().flush();
>   }
> {code}
> For {{HTable#put}}
> {code}
>   public void put(final Put put) throws IOException {
>     getBufferedMutator().mutate(put);
>     flushCommits();
>   }
> {code}
> If we launch multiple threads to put in parallel, below sequence might happen because
{{HTable#getBufferedMutator}} is not thread safe:
> {noformat}
> 1. ThreadA runs to getBufferedMutator and finds mutator==null
> 2. ThreadB runs to getBufferedMutator and finds mutator==null
> 3. ThreadA initialize mutator to instanceA, then calls mutator#mutate,
> adding one put (putA) into {{writeAsyncBuffer}}
> 4. ThreadB initialize mutator to instanceB
> 5. ThreadA runs to flushCommits, now mutator is instanceB, it calls
> instanceB's flush method, putA is lost
> {noformat}
> After fixing this, we will find quite some contention on {{BufferedMutatorImpl#flush}},
so more efforts required to make HTable thread safe but with good performance meanwhile.

This message was sent by Atlassian JIRA

View raw message