hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ChiaPing Tsai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17174) Use shared threadpool and AsyncProcess in BufferedMutatorImpl
Date Wed, 30 Nov 2016 13:06:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15708538#comment-15708538
] 

ChiaPing Tsai commented on HBASE-17174:
---------------------------------------

Two new public API.
# Connection#createAsyncProcess(): instantiating the AP
# BufferedMutatorParams#setAsyncProcess()/getAsyncProcess(): User can pass the reuse AP to
different BF via BufferedMutatorParams if they want control the traffic across the tables.

These changes are all in the hbase-client module, so it may introduce some compatibility issue
in the future.

Any feedback is welcome.


> Use shared threadpool and AsyncProcess in BufferedMutatorImpl
> -------------------------------------------------------------
>
>                 Key: HBASE-17174
>                 URL: https://issues.apache.org/jira/browse/HBASE-17174
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 2.0.0
>            Reporter: ChiaPing Tsai
>            Assignee: ChiaPing Tsai
>            Priority: Minor
>             Fix For: 2.0.0
>
>         Attachments: HBASE-17174.v0.patch, HBASE-17174.v1.patch, HBASE-17174.v2.patch,
HBASE-17174.v3.patch, HBASE-17174.v4.patch, HBASE-17174.v5.patch
>
>
> The following are reasons of reusing the pool and AP.
> # A update-heavy application, for example, loader, creates many BufferedMutator for batch
updates. But these BufferedMutators can’t share a large threadpool because the shutdown()
method will be called when closing any BufferedMutator. This patch adds a flag into BufferedMutatorParams
for preventing calling the shutdown() method in BufferedMutatorImpl#close
> # The AsyncProcess has the powerful traffic control, but the control is against the single
table currently. Because the AP is constructed at BufferedMutatorImpl's construction. This
patch change the BufferedMutatorImpl's construction for reuse the AP so that the updates for
different tables can be restricted by the same AP.
> Additionally, there are two changes(aren't included in latest patch) for #2.
> 1) The AP will be exposed to user.
> 2) A new method will be added to Connection for instantiating a AP.
> All suggestions are welcome.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message