hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Liochon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7247) Assignment performances decreased by 50% because of regionserver.OpenRegionHandler#tickleOpening
Date Thu, 04 Apr 2013 18:09:16 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13622595#comment-13622595
] 

Nicolas Liochon commented on HBASE-7247:
----------------------------------------

I'm doing some tests on trunk on recovery.

Test is:
 - 1000 empty regions on one RS
 - on another computer, all services are running: master, ZK, second RS.
 - Stop (cleanly) the first RS.

Recovery (in this case assignment only) takes 2 minutes.
All the time seems to be spent in writing/reading to ZK.

We have ~3000 calls to ZK#transitionNode, including 1000 for tickleOpening.

Each transition is:
 sync
 readData
 setData


Commenting the sync makes no difference (it could be an effect of the test env).
removing tickleOpening brings the result to 80 seconds (vs. 120s)
Checking but not writing in tickleOpening puts us around 85s. I will do a proper patch with
this.
                
> Assignment performances decreased by 50% because of regionserver.OpenRegionHandler#tickleOpening
> ------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-7247
>                 URL: https://issues.apache.org/jira/browse/HBASE-7247
>             Project: HBase
>          Issue Type: Improvement
>          Components: master, Region Assignment, regionserver
>    Affects Versions: 0.96.0
>            Reporter: Nicolas Liochon
>            Assignee: Nicolas Liochon
>             Fix For: 0.95.1
>
>         Attachments: 7247.v1.patch
>
>
> The regionserver.OpenRegionHandler#tickleOpening updates the region znode as "Do this
so master doesn't timeout this region-in-transition.".
> However, on the usual test, this makes the assignment time of 1500 regions goes from
70s to 100s, that is, we're 50% slower because of this.
> More generally, ZooKeper commits to disk all the data update, and this takes time. Using
it to provide a keep alive seems overkill. At the very list, it could be made asynchronous.
> I'm not sure how necessary these updates are required (I need to go deeper in the internal,
feedback welcome), but it seems very important to optimize this... The trival fix would be
to make this optional.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message