hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shreepadma Venugopalan" <shreepa...@cloudera.com>
Subject Re: Review Request: HIVE-3443: Add serdeParamKey option to Hive Metatool
Date Thu, 13 Sep 2012 16:52:54 GMT


> On Sept. 11, 2012, 12:38 a.m., Carl Steinbach wrote:
> > metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaTool.java, line
114
> > <https://reviews.apache.org/r/6962/diff/3/?file=152381#file152381line114>
> >
> >     Please add bad URI records for the different fields and verify that the metatool
takes a best-effort approach to completing the update operation.

I've added records with bad URI for avro.schema.url. Its not feasible to add bad records for
other field - metastore validates the URL and raises an exception. However I manually inserted
bad records in the metastore through SQL and verified that the metatool completes.


- Shreepadma


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6962/#review11291
-----------------------------------------------------------


On Sept. 13, 2012, 4:50 p.m., Shreepadma Venugopalan wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/6962/
> -----------------------------------------------------------
> 
> (Updated Sept. 13, 2012, 4:50 p.m.)
> 
> 
> Review request for hive, Carl Steinbach and Carl Steinbach.
> 
> 
> Description
> -------
> 
> This patch adds an additional option to Hive Metatool that allows Metatool to take in
a serdeParamKey from the user. Avro Serde's schema URL key used to be called schema.url in
the past whereas its called avro.schema.url now. The purpose of the patch is to make Metatool
more generic than what it is today so that its in a position to handle variations such as
the one described above. The new option looks as below,
> 
> -serdeParamKey <serde_param_key>=<value>
> 
> Note that the new Option -serdeParamKey is valid only with the -updateLocation option.
When the user attempts to use the serdeParamKey option with other options, an error is raised
and the usage is printed.
> 
> If the user doesn't pass -serdeParamKey as part of the -updateLocation option, Hive Metatool
searches for records with both "avro.schema.url" and "schema.url" keys and updates them.
> 
> 
> This addresses bug HIVE-3343.
>     https://issues.apache.org/jira/browse/HIVE-3343
> 
> 
> Diffs
> -----
> 
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 251d4ba 
>   metastore/src/java/org/apache/hadoop/hive/metastore/tools/HiveMetaTool.java a76594a

>   metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaTool.java 5790ae9 
> 
> Diff: https://reviews.apache.org/r/6962/diff/
> 
> 
> Testing
> -------
> 
> HiveMetaTool has been tested to verify that it handles both avro.schema.url and schema.url
correctly. Existing test case in TestHiveMetaTool.java has been modified to use AvroSerDe
instead of LazySimpleSerDe.
> 
> 
> Thanks,
> 
> Shreepadma Venugopalan
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message