hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Lawlor (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-6778) Deprecate Chore; its a thread per task when we should have one thread to do all tasks
Date Wed, 14 Jan 2015 18:54:35 GMT

     [ https://issues.apache.org/jira/browse/HBASE-6778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Lawlor updated HBASE-6778:
-----------------------------------
    Attachment: thread_dump_HMaster.local.out

I started to look into this issue this week. Initial investigation revealed that on average
the number of threads running within the HMaster process that can be attributed to Chores
is around 12 threads. I have attached a sample thread dump above (you can see the active Chore
threads by searching for 'Chore').

As discussed, the goal of a proposed solution should be to reduce the number of running threads
by allowing the running Chores to share threads that are idle in between Chore cycles. The
idea of using ScheduledThreadPoolExecutor seems well suited for the job. 

A ScheduleThreadPoolExecutor allows us to specify the maximum number of threads that we want
to allow the running Chores to use. Since there may be Chores with tasks that take a while
to execute, this value would have to be dynamic so that it could increase/decrease during
varying Chore loads. Specifically, the executor needs to account for the worst case scenario
that all running Chores have tasks that take longer than their period (i.e. the chore finishes
after its next scheduled execution should have started). Thus the core pool size could be
set to a small initial value, and if we recognize that Chores are missing their scheduled
start times, we could increase the core pool size to a maximum of however many active chores
there are.

In summary, I am proposing that we design a service that acts as an interface to an underlying
ScheduleThreadPoolExecutor whose core pool size can vary. Any feedback on this proposed solution
and where this may fit into the HBase workflow would be greatly appreciated.

> Deprecate Chore; its a thread per task when we should have one thread to do all tasks
> -------------------------------------------------------------------------------------
>
>                 Key: HBASE-6778
>                 URL: https://issues.apache.org/jira/browse/HBASE-6778
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: Jonathan Lawlor
>         Attachments: thread_dump_HMaster.local.out
>
>
> Should use something like ScheduledThreadPoolExecutor instead (Elliott said this first
I think; J-D said something similar just now).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message