hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4742) Split dead server's log in parallel
Date Sun, 06 Nov 2011 00:28:53 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144877#comment-13144877

Phabricator commented on HBASE-4742:

Kannan has commented on the revision "[jira] [HBASE-4742] Split dead server's log in parallel".

  In practice, the actual log splitting will happen in region server... so at the master level
it is only going to consume as many threads as concurrently dead region servers... but the
threads won't really be doing much expensive work. I would imagine it would to spawn say even
1000 threads without much adverse affect.

  Having said that, if region servers are repeatedly starting/dying etc. it could end in some
degenerate cases where things could grow unbounded and take the master down. So limiting this
seems the safer thing to do.

  Liyin: In practice, while I think what you have is just fine and unlikely to cause any serious
issues... does it complicate the design too much to back this by a threadpool backed with
a queue and set the limit pretty high to say 1000? If the changes are not too intrusive, I
would suggest doing the threadpool. If more involved, we can layer that in in a second pass.



> Split dead server's log in parallel
> -----------------------------------
>                 Key: HBASE-4742
>                 URL: https://issues.apache.org/jira/browse/HBASE-4742
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Liyin Tang
>            Assignee: Liyin Tang
>         Attachments: D237.1.patch, D237.2.patch, D237.3.patch, D237.4.patch
> When one region server goes down, the master will shutdown the region server and split
its log.
> However, splitting log is a blocking call and it would take some time.
> If more than one region server go down, the master will split its log one by one, which
is not efficient.
> Since we have the distributed log split, we could split these logs from the dead servers
in parallel. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message