hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jurriaan Mous (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-12736) Let MetaScanner recycle a given Connection
Date Tue, 23 Dec 2014 15:01:13 GMT

     [ https://issues.apache.org/jira/browse/HBASE-12736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jurriaan Mous updated HBASE-12736:
    Attachment: HBASE-12736-v4.patch

You are right. It uses way less Threads in general. I was too focussed on the graph.
I was worried about the more active threads and more memory usage. But it gets all cleaned
up and they end up the same.

Looking into what was causing it it was pretty simple. Since all MetaScanners use the same
connection, they use the same thread pool. And now each scanner does not close the connection
and thus they don't clean up their threads along the way. And the thread pool limit is not
reached while scanning so the peaks are higher.

> Let MetaScanner recycle a given Connection
> ------------------------------------------
>                 Key: HBASE-12736
>                 URL: https://issues.apache.org/jira/browse/HBASE-12736
>             Project: HBase
>          Issue Type: Improvement
>          Components: Client
>            Reporter: Jurriaan Mous
>            Assignee: Jurriaan Mous
>         Attachments: HBASE-12736-v1.patch, HBASE-12736-v2.patch, HBASE-12736-v3.patch,
HBASE-12736-v4.patch, HBASE-12736.patch, profile_TestMetaScanner_after_patch.png, profile_TestMetaScanner_before_patch.png
> It is very heavy to create a Connection on each meta scan. Connections create RpcClients,
RpcClients create RPC connections and all cannot be recycled.
> Tests with a lot of metascans are very heavy with the async client.
> This issue is to make anything that uses metaScans reuse the same connection.

This message was sent by Atlassian JIRA

View raw message