hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Lawlor (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-6778) Deprecate Chore; its a thread per task when we should have one thread to do all tasks
Date Tue, 20 Jan 2015 21:13:35 GMT

     [ https://issues.apache.org/jira/browse/HBASE-6778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jonathan Lawlor updated HBASE-6778:
    Attachment: HBASE_6778_WIP_v1.patch

Work In Progress Update:
I have been working on implementing the outlined solution and have attached a patch representing
my progress thus far. 

As it stands, the TestChoreService class is very rough and represents tests that I wrote to
simply test a few underlying concepts. More extensive and robust test coverage will be implemented.

The two main classes that I have been developing are ScheduledChore.java and ChoreService.java:

Progress/Details with respect to ScheduledChore.java:
- The ScheduledChore.java was designed with the current Chore implementation in mind. I wanted
to maintain all of the existing functionality while making it compatible with the new workflow.

- A new concept that appears in ScheduledChore that is not present in the existing Chore implementation
is the concept of "missing start time". ScheduledChores track the time between successive
runs. If the time between successive runs exceeds 1.5 * period then we say that the chore
has missed its start time and we notify the ChoreService of this event. This is a useful workflow
as it helps the ChoreService decide when it needs to increase the number of threads.

Progress/Details with respect to ChoreService.java:
- The ChoreService.java class is the class that defines how Chores are scheduled and how the
thread count is controlled. 
- The service allows Chores to be scheduled, cancelled, rescheduled, and triggered immediately.
It also provides some useful methods for checking the status of scheduled chores.
- When chores are added and removed the ChoreService may make some decisions with respect
to the size of the core pool. For example, if a Chore that caused an increase in the core
pool size is removed from the service, an attempt is made to reduce the core pool size.

The finer grain details can be seen by going through the patch attached above. The next steps
are to refine the test coverage and begin converting existing Chores into ScheduledChores.

One design detail that I was hoping to get feedback on was where the instance of ChoreService
should live in the codebase. Is there an object that would be suitable to hold an instance
of ChoreService that could then be used to service objects that want to start a periodic chore?

All feedback is greatly appreciated.


> Deprecate Chore; its a thread per task when we should have one thread to do all tasks
> -------------------------------------------------------------------------------------
>                 Key: HBASE-6778
>                 URL: https://issues.apache.org/jira/browse/HBASE-6778
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: Jonathan Lawlor
>         Attachments: HBASE_6778_WIP_v1.patch, thread_dump_HMaster.local.out
> Should use something like ScheduledThreadPoolExecutor instead (Elliott said this first
I think; J-D said something similar just now).

This message was sent by Atlassian JIRA

View raw message