hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasad Mujumdar" <pras...@cloudera.com>
Subject Re: Review Request 14169: HIVE-3764: Support metastore version consistency check
Date Wed, 18 Sep 2013 05:45:51 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14169/
-----------------------------------------------------------

(Updated Sept. 18, 2013, 5:45 a.m.)


Review request for hive, Ashutosh Chauhan and Brock Noland.


Changes
-------

Changes per review feedback.
- rename the comment field.
- fixed typos and updated variable name
- fixed the version comparison case
- build script update to set the hive dist path for testing (was missed in the original diff
when the large patch was split into two.)


Bugs: HIVE-3764
    https://issues.apache.org/jira/browse/HIVE-3764


Repository: hive-git


Description
-------

This is a 0.12 specific patch. The trunk patch will include additional metastore scripts which
I will attach separately to the ticket.

- Added a new table in the metastore schema to store the Hive version in the metastore.
- Metastore handler compare the version stored in the schema with its own version. If there's
a mismatch, then it can either record the correct version or raise error. The behavior is
configurable via a new Hive config. This config when set, also restrict dataNucleus to auto
upgrade the schema.
- The new schema creation and upgrade scripts record the new version in the metastore version
table.
- Added 0.12 upgrade scripts for all supported DBs to creates the new table version tables
in 0.12 metastore schema

The current patch has the verification turned off by default. I would prefer to keep it enabled,
though it require any add-hoc setup to explicitly disable it (or create the metastore schema
by running scripts). The default can be changed or left as is as per the consensus. 


Diffs (updated)
-----

  build-common.xml ad5ac23 
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1474469 
  conf/hive-default.xml.template c123452 
  metastore/scripts/upgrade/derby/014-HIVE-3764.derby.sql PRE-CREATION 
  metastore/scripts/upgrade/derby/hive-schema-0.12.0.derby.sql cce544f 
  metastore/scripts/upgrade/derby/upgrade-0.10.0-to-0.11.0.derby.sql cae7936 
  metastore/scripts/upgrade/derby/upgrade-0.11.0-to-0.12.0.derby.sql 492cc93 
  metastore/scripts/upgrade/derby/upgrade.order.derby PRE-CREATION 
  metastore/scripts/upgrade/mysql/014-HIVE-3764.mysql.sql PRE-CREATION 
  metastore/scripts/upgrade/mysql/hive-schema-0.12.0.mysql.sql 22a77fe 
  metastore/scripts/upgrade/mysql/upgrade-0.11.0-to-0.12.0.mysql.sql 375a05f 
  metastore/scripts/upgrade/mysql/upgrade.order.mysql PRE-CREATION 
  metastore/scripts/upgrade/oracle/014-HIVE-3764.oracle.sql PRE-CREATION 
  metastore/scripts/upgrade/oracle/hive-schema-0.12.0.oracle.sql 85a0178 
  metastore/scripts/upgrade/oracle/upgrade-0.10.0-to-0.11.0.mysql.sql PRE-CREATION 
  metastore/scripts/upgrade/oracle/upgrade-0.11.0-to-0.12.0.oracle.sql a2d0901 
  metastore/scripts/upgrade/oracle/upgrade.order.oracle PRE-CREATION 
  metastore/scripts/upgrade/postgres/014-HIVE-3764.postgres.sql PRE-CREATION 
  metastore/scripts/upgrade/postgres/hive-schema-0.12.0.postgres.sql 7b319ba 
  metastore/scripts/upgrade/postgres/upgrade-0.11.0-to-0.12.0.postgres.sql 9da0a1b 
  metastore/scripts/upgrade/postgres/upgrade.order.postgres PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 39dda92 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreSchemaInfo.java PRE-CREATION

  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java a27243d 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java e410c3a 
  metastore/src/model/org/apache/hadoop/hive/metastore/model/MVersionTable.java PRE-CREATION

  metastore/src/model/package.jdo c42b5b0 
  metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java 8066784

  metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 0f9b16c

  metastore/src/test/org/apache/hadoop/hive/metastore/TestMetastoreVersion.java PRE-CREATION


Diff: https://reviews.apache.org/r/14169/diff/


Testing
-------

Added new tests for schema verification. Manually tested the upgrades using derby and MySQL.


Thanks,

Prasad Mujumdar


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message