hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 张铎(Duo Zhang) <palomino...@gmail.com>
Subject Re: [ANNOUNCE] Asynchronous client is available in HBase 2.0
Date Sun, 06 Aug 2017 00:19:18 GMT
The current pe tool, both our pe and the YCSB, are all designed for sync
client, so usually there will be no performance boost since in these tests
we can only use async client in a sync way. And if we do some simple
modifications, such as do not wait reply and keep sending requests, no
doubt async client will be much faster but this is not fair for sync client.

Maybe we could reimplement hbase-rest with async client and compare its
performance with the old implementation? As I said before, proxy like
service is always the best candidate for using async client.

Thanks.

2017年8月4日星期五,Guanghao Zhang <zghaobac@gmail.com> 写道:

> @stack, sorry for reply late, sir. As HBASE-16388
> <https://issues.apache.org/jira/browse/HBASE-16388> said, the client
> threads may be blocked by only one slow region server. But a better
> solution for this problem is to use async hbase client. And there are many
> reasons to make a region server slow, slow read from hdfs, gc stw, etc. So
> this should be a advantage of the async client. PS: my colleague Zheng Hu
> talked about this problem on today's HBaseCon Asia :-). About the
> performance of async client, there are two important metrics: latency and
> QPS. Our PE tool can be used to test latency and it will be addressed by
> HBASE-17994 <https://issues.apache.org/jira/browse/HBASE-17994>. And YCSB
> maybe better to test QPS. I will publish the test result when we get more
> test data. Thanks.
>
>
> 2017-07-22 0:35 GMT+08:00 Andrew Purtell <apurtell@apache.org
> <javascript:;>>:
>
> > This is exciting!
> > Thanks so much for this hard work.
> >
> > On Fri, Jul 21, 2017 at 2:21 AM, Phil Yang <ud1937@gmail.com
> <javascript:;>> wrote:
> >
> > > I filed an issue HBASE-18334
> > > <https://issues.apache.org/jira/browse/HBASE-18334> to remove the old
> > sync
> > > implementation. I think we can do it in 3.0 and keep sync logic in
> > > branch-2? Of course we can also remove it in 2.1 or higher 2.x branch
> but
> > > if we remove it since 2.0 it may be a little radical  :)
> > >
> > > Thanks,
> > > Phil
> > >
> > >
> > > 2017-07-21 6:15 GMT+08:00 Stack <stack@duboce.net <javascript:;>>:
> > >
> > > > Very sweet Guanghao. Thanks for all the hard work (you, Duo and all
> who
> > > > contributed to the async work).
> > > >
> > > > When you think we should move the sync client to be on top of the
> async
> > > > chassis?
> > > >
> > > > Thanks,
> > > > St.Ack
> > > >
> > > > P.S. It looks like you updated its status in this doc,
> > > > https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_
> > > > ktczrlKHK8N4SZzs/edit#,
> > > > thanks.
> > > >
> > > > On Thu, Jul 20, 2017 at 4:51 AM, Guanghao Zhang <zghaobac@gmail.com
> <javascript:;>>
> > > > wrote:
> > > >
> > > > > Dear all,
> > > > >
> > > > > Asynchronous client aims to provide the ability to access HBase
> > > > > asynchronously. You can obtain an AsyncConnection from
> > > ConnectionFactory,
> > > > > and then get an asynchronous table instance (for DML operations)
or
> > an
> > > > > asynchronous admin instance (for DDL operations) from it to access
> > > HBase.
> > > > > For the asynchronous table or admin, most methods have the same
> > meaning
> > > > > with the old Table or Admin interface, expect that the return value
> > is
> > > > > wrapped with a CompletableFuture (java 8 or higher is required)
> > > usually.
> > > > > You can get more details from the documentation[1][2].
> > > > >
> > > > > There are two examples about the asynchronous client.
> > > > AsyncClientExample[3]
> > > > > is a simple example to show you how to use AsyncTable.
> > > > HttpProxyExample[4]
> > > > > is an example for advance user to show you how to use RawAsyncTable
> > to
> > > > > write a fully asynchronous HTTP proxy server. Welcome to try the
> > > > > asynchronous client in your project.
> > > > >
> > > > > We tested the performance of asynchronous client by
> > > > > the PerformanceEvaluation tool. And it showed that async client has
> > > > almost
> > > > > same latency with sync client[5]. Notice: when we did performance
> > test
> > > > for
> > > > > asynchronous client. We found some performance issue when use jdk
> > > > 1.8_111.
> > > > > And it was fixed when use jdk 1.8_131 [6]. So suggest use a bigger
> > > > version
> > > > > than jdk8_131 when you use the asynchronous client.
> > > > >
> > > > > The major umbrella issues include:
> > > > > HBASE-16432 Revisit the asynchronous ipc implementation[7]
> > > > > HBASE-16833 Implement asynchronous hbase client based on
> > HBASE-15921[8]
> > > > > HBASE-17359 Implement async admin[9]
> > > > >
> > > > > There is also an issue HBASE-17856 to track the further
> improvements
> > > for
> > > > > asynchronous hbase client[10].
> > > > >
> > > > > Finally, thanks Duo Zhang and Zheng Hu who worked with me together
> to
> > > > > implement the asynchronous client.
> > > > >
> > > > > Best Regards,
> > > > > Guanghao
> > > > >
> > > > > [1] https://hbase.apache.org/book.html#async.client
> > > > > [2] https://hbase.apache.org/book.html#async.admin
> > > > > [3]
> > > > > https://github.com/apache/hbase/blob/master/hbase-
> > > > > examples/src/main/java/org/apache/hadoop/hbase/client/
> > > > > example/AsyncClientExample.java
> > > > > [4]
> > > > > https://github.com/apache/hbase/blob/master/hbase-
> > > > > examples/src/main/java/org/apache/hadoop/hbase/client/
> > > > > example/HttpProxyExample.java
> > > > > [5] https://www.slideshare.net/HBaseCon/hbase-at-xiaomi
> > > > > [6]
> > > > > http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/
> > 687fd7c7986d/src/share/
> > > > > classes/java/util/concurrent/CompletableFuture.java
> > > > > [7] https://issues.apache.org/jira/browse/HBASE-16432
> > > > > [8] https://issues.apache.org/jira/browse/HBASE-16833
> > > > > [9] https://issues.apache.org/jira/browse/HBASE-17359
> > > > > [10] https://issues.apache.org/jira/browse/HBASE-17856
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> >    - A23, Crosstalk
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message