hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shreepadma Venugopalan" <shreepa...@cloudera.com>
Subject Re: Review Request: HIVE-3056: Ability to bulk update location field in Db/Table/Partition records
Date Wed, 29 Aug 2012 03:24:39 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6650/
-----------------------------------------------------------

(Updated Aug. 29, 2012, 3:24 a.m.)


Review request for hive and Carl Steinbach.


Changes
-------

This revision to the original patch addresses almost all of the review comments from revision
# 5. Additionally the following changes have been made to the -updateLocation option,

* A new option -dryRun has been added. When run with the -dryRun option persistent changes
are not made. Instead the current location and the proposed new location are printed to stdout.
* Both new-loc and old-loc have to be valid URIs. This validation is performed in the code
and an error is raised if either the new-loc or the old-loc is not a valid URI. Please note
that both the host name and scheme fields of the URI are required, while the port is optional.
However, if the old-loc contains a port, the new-loc should specify a port too. The motivation
behind making the scheme a required field is to prevent an inadvertent update of the location
field when the schemes doesn't match. Please note that the primary motivation for this tool
at this point is to update the location field to ensure Hive survives a non-HA to HA upgrade
and vice versa.
* The test case has been fixed to remove hard coded paths in the locationURI field of MDatabase.


Additionally the updateLocation option of the metatool has been manually tested on a real
cluster. All of the locations in the metastore correctly point to the HDFS NN and running
metatool -updateLocation <new-nn-loc> <old-nn-loc> correctly updates the location
of all relevant records to point to the new HA NN.


Description
-------

This patch implement hive metatool which,

* lets admins perform a HA upgrade by patching the location of the NN in Hive's metastore
* allows JDOQL to be executed against the metastore.


This addresses bug HIVE-3056.
    https://issues.apache.org/jira/browse/HIVE-3056


Diffs (updated)
-----

  bin/ext/metatool.sh PRE-CREATION 
  bin/metatool PRE-CREATION 
  build.xml 6712af9 
  eclipse-templates/TestHiveMetaTool.launchtemplate PRE-CREATION 
  metastore/ivy.xml 3011d2f 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 045b550 
  metastore/src/java/org/apache/hadoop/hive/metastore/tools/HiveMetaTool.java PRE-CREATION

  metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaTool.java PRE-CREATION 

Diff: https://reviews.apache.org/r/6650/diff/


Testing
-------

A new JUnit test - TestHiveMetaTool - has been added to test the various metatool options.


Thanks,

Shreepadma Venugopalan


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message