hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Himanshu Vashishtha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8348) Polish the migration to 0.96
Date Tue, 17 Sep 2013 18:40:51 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769793#comment-13769793
] 

Himanshu Vashishtha commented on HBASE-8348:
--------------------------------------------

So, I tested migration to 0.96 on a production quality (dirty) dataset (got a dataset dump
from a 15 nodes cluster). 
Following are some numbers.

h5.Dataset:
1) Tables: 100
2) Number of regions: 743
3) RS on original cluster: 15
4) Logs to split: 425 wal files, 26G
5) Number of HFiles to detect for HFileV1: ~ 4k
6) Snapshots: 4

h5. Upgrade numbers:
1) Time for hfilev1 checker: ~ 15s
2) Time to do log split (offline mode): 900s (15min)

The destination cluster is 4 RS + 1 master. 
Looking at the logs, cluster UI, the upgrade is successful.

It is to note that for upgrade, we recommend clean shutdown of the 94.x cluster; the above
test is an example of a very bad shutdown (425 WAL files across 15 regionservers to split).
                
> Polish the migration to 0.96
> ----------------------------
>
>                 Key: HBASE-8348
>                 URL: https://issues.apache.org/jira/browse/HBASE-8348
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.95.0
>            Reporter: Jean-Daniel Cryans
>            Assignee: rajeshbabu
>            Priority: Blocker
>             Fix For: 0.96.0
>
>         Attachments: 8348v5.txt, 8348v5.txt, HBASE-8348-approach-2.patch, HBASE-8348-approach-2-v2.1.patch,
HBASE-8348-approach-2-v2.2.patch, HBASE-8348-approach-2-v2.3.patch, HBASE-8348-approach-2-v2.4.patch,
HBASE-8348-approach-3.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch,
log, log-2, Upgradeto96.docx, Upgradeto96.pdf
>
>
> Currently, migration works but there's still a couple of rough edges:
>  - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on
the filesystem.
>  - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete
it ourselves.
>  - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone
now.
> Elliott was also mentioning that we could have "hbase migrate" do the HFileV1 checks,
clear ZK, remove ROOT, etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message