hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Szita <sz...@cloudera.com>
Subject Re: Review Request 56546: Allow user to update AVRO table schema via command even if table has external schema
Date Wed, 22 Feb 2017 11:56:47 GMT


> On Feb. 21, 2017, 10:23 p.m., Aihua Xu wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 1207
> > <https://reviews.apache.org/r/56546/diff/1/?file=1629525#file1629525line1207>
> >
> >     Should the command like 'alter table add column' always alter the avro schema?
Just wondering if it's necessary to add this configuration.
> >     
> >     So what we will do is to update the schema file?

I think this has to be optional.
The goal of the patch is to enable alterting tables that were defined using avro schema url
or literal. Currently this doesn't work in Hive.
If an avro table was created using a schema file, it will have to drop its avro.schema.url
tbl property during alter table command. We cannot know if for example this file has any other
use case, so we cannot touch it. "Detaching" the table schema from the schema file however
might be unexpected to some users, hence the configurability.


> On Feb. 21, 2017, 10:23 p.m., Aihua Xu wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java, line 3568
> > <https://reviews.apache.org/r/56546/diff/1/?file=1629526#file1629526line3568>
> >
> >     This is avro specific call. I'm wondering if we can achieve this by something
like, giving SerDe a function handleAddColumn(). Not sure if it makes sense. Can you take
a look?

I see your point, however this a single case. I would rather make an abstract handleAddColumn()
method if in the future we find that other serdes could make use of it.


- Adam


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56546/#review166239
-----------------------------------------------------------


On Feb. 22, 2017, 11:48 a.m., Adam Szita wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56546/
> -----------------------------------------------------------
> 
> (Updated Feb. 22, 2017, 11:48 a.m.)
> 
> 
> Review request for hive, Aihua Xu, Peter Vary, and Sergio Pena.
> 
> 
> Bugs: HIVE-13780
>     https://issues.apache.org/jira/browse/HIVE-13780
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Allow user to update AVRO table schema via command even if table's definition was defined
through schema file / literal in tblproperties
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 3777fa96ba8eac79e07e454ee437fb05583158c5

>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java adabe70fa8f0fe1b990c6ac578a14ff5af06fc93

>   ql/src/test/queries/clientnegative/avro_add_column_extschema.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/avro_add_column_extschema.q PRE-CREATION 
>   ql/src/test/results/clientnegative/avro_add_column_extschema.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/avro_add_column_extschema.q.out PRE-CREATION 
>   serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java f18585da1d108abdd500437362eb388b21030ec7

> 
> Diff: https://reviews.apache.org/r/56546/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Adam Szita
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message