hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Marc Spaggiari (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17361) Make HTable thread safe
Date Sat, 24 Dec 2016 13:48:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15774924#comment-15774924
] 

Jean-Marc Spaggiari commented on HBASE-17361:
---------------------------------------------

So previously the recommendation was to have one single connection (or a couple, pooled) and
get Table objects from there, one per thread.  Does it means it will not be the case anymore
in 2.0 and that we can directly pool again the table objects?

> Make HTable thread safe
> -----------------------
>
>                 Key: HBASE-17361
>                 URL: https://issues.apache.org/jira/browse/HBASE-17361
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Yu Li
>            Assignee: Yu Li
>         Attachments: HBASE-17361.patch, HBASE-17361.patch
>
>
> Currently HTable is marked as NOT thread safe, and this JIRA target at improving this
to take better usage of the thread-safe BufferedMutator.
> Some findings/work done:
> If we try to do put to the same HTable instance in parallel, there'll be problem, since
now we have {{HTable#getBufferedMutator}} like
> {code}
>    BufferedMutator getBufferedMutator() throws IOException {
>      if (mutator == null) {
>       this.mutator = (BufferedMutatorImpl) connection.getBufferedMutator(
>           new BufferedMutatorParams(tableName)
>               .pool(pool)
>               .writeBufferSize(connConfiguration.getWriteBufferSize())
>               .maxKeyValueSize(connConfiguration.getMaxKeyValueSize())
>       );
>     }
>     mutator.setRpcTimeout(writeRpcTimeout);
>     mutator.setOperationTimeout(operationTimeout);
>     return mutator;
>   }
> {code}
> And {{HTable#flushCommits}}:
> {code}
>   void flushCommits() throws IOException {
>     if (mutator == null) {
>       // nothing to flush if there's no mutator; don't bother creating one.
>       return;
>     }
>     getBufferedMutator().flush();
>   }
> {code}
> For {{HTable#put}}
> {code}
>   public void put(final Put put) throws IOException {
>     getBufferedMutator().mutate(put);
>     flushCommits();
>   }
> {code}
> If we launch multiple threads to put in parallel, below sequence might happen because
{{HTable#getBufferedMutator}} is not thread safe:
> {noformat}
> 1. ThreadA runs to getBufferedMutator and finds mutator==null
> 2. ThreadB runs to getBufferedMutator and finds mutator==null
> 3. ThreadA initialize mutator to instanceA, then calls mutator#mutate,
> adding one put (putA) into {{writeAsyncBuffer}}
> 4. ThreadB initialize mutator to instanceB
> 5. ThreadA runs to flushCommits, now mutator is instanceB, it calls
> instanceB's flush method, putA is lost
> {noformat}
> After fixing this, we will find quite some contention on {{BufferedMutatorImpl#flush}},
so more efforts required to make HTable thread safe but with good performance meanwhile.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message