hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4388) Second start after migration from 90 to trunk crashes
Date Sat, 22 Oct 2011 20:58:34 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13133478#comment-13133478

jiraposter@reviews.apache.org commented on HBASE-4388:

This is an automatically generated e-mail. To reply, visit:

(Updated 2011-10-22 20:59:14.138541)

Review request for hbase.


Address Ted feedback (I don't include tgz file in this diff)


Changes name of column in root that holds current version of the meta table from info:metamigrated
to info:v.

Current version of meta table is 0 (short).  The absence of the column means table has not
been migrated.

Upped version on HRegionInfo and made its readFields able to deserialze the older pre-0.92

Change MetaReader#putToCatalogTable so it no longer takes a region name.  Was redundant (needed
this method for migration).

Removed a bunch of MetaMigrationRemovingHTD.  Not needed any more or was duplicated code and
I axed it.

Found a bug in that we didn't migrate info:splitA or info:splitB if present -- i.e. if we
had offlined parent from split going on when migration was happening, we'd fail.

Migration is more idempotenty now.  We look at the HRI bytes and if older version, we'll update
(Removed the try to deserialize and catch exception stuff).

Moved methods from HBaseTestingUtility and from Writables that had to do w/ migration exclusively,
out and into MetaMigrationRemoveHTD migrating class (or into migration tests) so migration
code all in one place rather than spread about (more to do)

Changed MetaReader#isMetaTable to MetaReader#isRootTableRow.  Makes more sense now (needed

Renamed TestMetaMigration as TestMetaMigrationRemovingHTD.  Also, made this test startup on
top of a 0.90 hbase.rootdir to test it can migrate.

Added TestMigrationFrom090To092

This addresses bug hbase-4388.

Diffs (updated)

  src/main/java/org/apache/hadoop/hbase/HConstants.java 144f526 
  src/main/java/org/apache/hadoop/hbase/HRegionInfo.java e950df3 
  src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java 6387df9 
  src/main/java/org/apache/hadoop/hbase/catalog/MetaMigrationRemovingHTD.java 0c1d221 
  src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java f7f4d60 
  src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 91abb00 
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 12dfbcd 
  src/main/java/org/apache/hadoop/hbase/util/Writables.java f595af5 
  src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 6af3d35 
  src/test/java/org/apache/hadoop/hbase/client/TestMetaMigration.java 6cbf88c 
  src/test/java/org/apache/hadoop/hbase/client/TestMetaMigrationRemovingHTD.java PRE-CREATION

  src/test/java/org/apache/hadoop/hbase/migration/TestMigrationFrom090To092.java PRE-CREATION

Diff: https://reviews.apache.org/r/2539/diff


Migration tests pass.



> Second start after migration from 90 to trunk crashes
> -----------------------------------------------------
>                 Key: HBASE-4388
>                 URL: https://issues.apache.org/jira/browse/HBASE-4388
>             Project: HBase
>          Issue Type: Bug
>          Components: master, migration
>    Affects Versions: 0.92.0
>            Reporter: Todd Lipcon
>            Assignee: stack
>            Priority: Blocker
>             Fix For: 0.92.0
>         Attachments: 4388-v2.txt, 4388-v3.txt, 4388-v4.txt, 4388.txt, hbase-4388-root.dir.tgz,
hbase-master-nase.log, meta.tgz
> I started a trunk cluster to upgrade from 90, inserted a ton of data, then did a clean
shutdown. When I started again, I got the following exception:
> 11/09/13 12:29:09 INFO master.HMaster: Meta has HRI with HTDs. Updating meta now.
> 11/09/13 12:29:09 FATAL master.HMaster: Unhandled exception. Starting shutdown.
> java.lang.NegativeArraySizeException: -102
>         at org.apache.hadoop.hbase.util.Bytes.readByteArray(Bytes.java:147)
>         at org.apache.hadoop.hbase.HTableDescriptor.readFields(HTableDescriptor.java:606)
>         at org.apache.hadoop.hbase.migration.HRegionInfo090x.readFields(HRegionInfo090x.java:641)
>         at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:133)
>         at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:103)
>         at org.apache.hadoop.hbase.util.Writables.getHRegionInfoForMigration(Writables.java:228)
>         at org.apache.hadoop.hbase.catalog.MetaEditor.getHRegionInfoForMigration(MetaEditor.java:350)
>         at org.apache.hadoop.hbase.catalog.MetaEditor$1.visit(MetaEditor.java:273)
>         at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:633)
>         at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:255)
>         at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:235)
>         at org.apache.hadoop.hbase.catalog.MetaEditor.updateMetaWithNewRegionInfo(MetaEditor.java:284)
>         at org.apache.hadoop.hbase.catalog.MetaEditor.migrateRootAndMeta(MetaEditor.java:298)
>         at org.apache.hadoop.hbase.master.HMaster.updateMetaWithNewHRI(HMaster.java:529)
>         at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:472)
>         at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:309)

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message