hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matteo Bertozzi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8720) Only one snapshot region tasks that can run at a time
Date Wed, 12 Jun 2013 16:08:21 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681357#comment-13681357
] 

Matteo Bertozzi commented on HBASE-8720:
----------------------------------------

The current snapshot implementation is just a first cut, so some names and code are there
ready for the future. (Example, this concurrent name, the restoreWALs() in the restore code
and so on)

again, I don't think that having N idle thread around for the snapshot is a good idea, changing
the queue type can be a more acceptable patch, but still. If you don't have a test and a benchmark
that demonstrate that you have a real performance gain having more thread doesn't make sense
at this time. 
You may consider introducing a a new snapshot procedure that doesn't flush the memstore and
and keeps track of the logs, and with that you may consider increasing the number of threads,
since is more likely to have a perf gain and less downside.
                
> Only one snapshot region tasks that can run at a time
> -----------------------------------------------------
>
>                 Key: HBASE-8720
>                 URL: https://issues.apache.org/jira/browse/HBASE-8720
>             Project: HBase
>          Issue Type: Bug
>          Components: snapshots
>    Affects Versions: 0.94.8, 0.95.0
>            Reporter: binlijin
>         Attachments: 8720-v2.txt, HBASE-8720.patch
>
>
> {code}
>     SnapshotSubprocedurePool(String name, Configuration conf) {
>       // configure the executor service
>       long keepAlive = conf.getLong(
>         RegionServerSnapshotManager.SNAPSHOT_TIMEOUT_MILLIS_KEY,
>         RegionServerSnapshotManager.SNAPSHOT_TIMEOUT_MILLIS_DEFAULT);
>       int threads = conf.getInt(CONCURENT_SNAPSHOT_TASKS_KEY, DEFAULT_CONCURRENT_SNAPSHOT_TASKS);
>       this.name = name;
>       executor = new ThreadPoolExecutor(1, threads, keepAlive, TimeUnit.MILLISECONDS,
>           new LinkedBlockingQueue<Runnable>(), new DaemonThreadFactory("rs("
>               + name + ")-snapshot-pool"));
>       taskPool = new ExecutorCompletionService<Void>(executor);
>     }
> {code}
> ThreadPoolExecutor: 
> corePoolSize:1
> maximumPoolSize:3
> workQueue:LinkedBlockingQueue,unlimited
> so when a new task submit to the ThreadPoolExecutor, if there is a task is running, the
new task is queued in the queue, so all snapshot region tasks execute one by one.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message