hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billy Pearson (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-722) Shutdown and Compactions
Date Tue, 08 Jul 2008 22:16:31 GMT

    [ https://issues.apache.org/jira/browse/HBASE-722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12611828#action_12611828
] 

Billy Pearson commented on HBASE-722:
-------------------------------------

I thank we have always queue up the compaction's on start up. might check but I have always
seen this when start with a lot of regions needing compaction. so I thank that part is fine.

The part on shutdown the simplest solution above is the second one 
Empty the compaction queue so it has no other regions to compact when done with the current
one on shutdown message received.
I thank that would be a simple fix and should workout fine for stopping the compaction of
more then the current compaction on shutdown.

> Shutdown and Compactions
> ------------------------
>
>                 Key: HBASE-722
>                 URL: https://issues.apache.org/jira/browse/HBASE-722
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: Billy Pearson
>            Priority: Trivial
>             Fix For: 0.3.0
>
>
> Currently I thank the region server closes the region spits in a set order this same
order is used in the compaction/spit thread after a startup and all regions need compaction.
> This causes the shutdown time to take a vary long time depending on what region the server
is compacting.
> I have seen my cluster take more then 60 mins to  shutdown waiting for several regions
to finish compaction.
> The problem I am seeing is say the compaction thread is working on a on region 2 of 4
in the list.
> When the region server get the  MSG_REGIONSERVER_QUIESCE from the master it closes in
order from 1-4
> So it closes region 1 and waits for region 2 to finish compaction before closing it.
> The problem is it never closes region split 3 or 4 while waiting for region 2 to complete
the compaction.
> So example say 3 and 4 regions are waiting to be compacted also. When region 2 is done
region 3 start compaction within milliseconds of 2's finished compaction.
> This happens faster then the region server can close region 3 so the region server hangs
around compacting regions until it run out of open regions needing compact.
> With a lot of regions this could hang some region server a long time to Shutdown the
cluster.
> Solutions 1 or 2 below will work I thank
> 1. Close all regions not currently getting compaction so when the compaction thread complete
it has no other regions open to compact 
> 2. Empty the compaction que so it has no other regions to compact when done with the
current one.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message