hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phil Yang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-16505) Add AsyncRegion interface to pass deadline and support async operations
Date Wed, 31 Aug 2016 13:55:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15452306#comment-15452306
] 

Phil Yang edited comment on HBASE-16505 at 8/31/16 1:54 PM:
------------------------------------------------------------

[~yuzhihong@gmail.com] [~carp84] Thanks for your reply, let's reach a consensus what this
interface will do first.

I think there are three goals for the AsyncRegion interface:

The first goal with highest priority is passing a context which let HRegion know what is the
deadline of this method calling, which will be used by HBASE-16492. We can set a timeout to
prevent our handlers being stuck on a blocking operation too long and wasting timeout on a
request that has been considered as timeout at client side.

The second goal with a lower priority is for SEDA at server side, this is not related with
RPC protocol with clients. In other words, when we move HBase to SDEA, clients still use current
protocol. And of course we can have AsyncTable at client, which is not related with SDEA at
server. They are dealing with different problems. SDEA uses a queue to connect different stages.
No matter how we design the new architecture, at least we need a stage to prepare the request,
one or more stages handling the request(the work in HRegion), and a stage to assemble the
response. Now they are in the same stage so the interface of Region is blocking. But when
we use different threads to do the work before the operation of Region and the work after
the operation of Region. The operation of Region must be async.

The third goal with lowest priority is async RPC protocol. We can writing response in a streaming
way which is very useful in Scan. This patch do not fully consider this, so in RegionOperationContext
we can only done the job with the whole result, doesn't support partial result. In the future
we can pass a streaming RPC context like grpc's StreamObserver in RegionOperationContext (for
example, the type of context may be RegionOperationContext<StreamObserver<Cell>>)
so we can write response in a streaming way.

Therefore, firstly, all methods which may be blocked for a while should have an async version
in AsyncRegion(Yes, waitForLock is not needed in AsyncRegion's getRowLock ). Secondly, in
this patch and in the future for a long period of time, we still write response when we complete
the whole request. So the logic of processRowsWithLocks is not related with clients. The time
of locking is not changed.


was (Author: yangzhe1991):
[~yuzhihong@gmail.com]] [~carp84] Thanks for your reply, let's reach a consensus what this
interface will do first.

I think there are three goals for the AsyncRegion interface:

The first goal with highest priority is passing a context which let HRegion know what is the
deadline of this method calling, which will be used by HBASE-16492. We can set a timeout to
prevent our handlers being stuck on a blocking operation too long and wasting timeout on a
request that has been considered as timeout at client side.

The second goal with a lower priority is for SEDA at server side, this is not related with
RPC protocol with clients. In other words, when we move HBase to SDEA, clients still use current
protocol. And of course we can have AsyncTable at client, which is not related with SDEA at
server. They are dealing with different problems. SDEA uses a queue to connect different stages.
No matter how we design the new architecture, at least we need a stage to prepare the request,
one or more stages handling the request(the work in HRegion), and a stage to assemble the
response. Now they are in the same stage so the interface of Region is blocking. But when
we use different threads to do the work before the operation of Region and the work after
the operation of Region. The operation of Region must be async.

The third goal with lowest priority is async RPC protocol. We can writing response in a streaming
way which is very useful in Scan. This patch do not fully consider this, so in RegionOperationContext
we can only done the job with the whole result, doesn't support partial result. In the future
we can pass a streaming RPC context like grpc's StreamObserver in RegionOperationContext (for
example, the type of context may be RegionOperationContext<StreamObserver<Cell>>)
so we can write response in a streaming way.

Therefore, firstly, all methods which may be blocked for a while should have an async version
in AsyncRegion(Yes, waitForLock is not needed in AsyncRegion's getRowLock ). Secondly, in
this patch and in the future for a long period of time, we still write response when we complete
the whole request. So the logic of processRowsWithLocks is not related with clients. The time
of locking is not changed.

> Add AsyncRegion interface to pass deadline and support async operations
> -----------------------------------------------------------------------
>
>                 Key: HBASE-16505
>                 URL: https://issues.apache.org/jira/browse/HBASE-16505
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Phil Yang
>            Assignee: Phil Yang
>         Attachments: HBASE-16505-v1.patch
>
>
> If we want to know the correct setting of timeout in read/write path, we need add a new
parameter in operation-methods of Region.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message