hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ferdinand Xu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10324) Hive metatool should take table_param_key to allow for changes to avro serde's schema url key
Date Thu, 23 Apr 2015 01:26:38 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14508270#comment-14508270
] 

Ferdinand Xu commented on HIVE-10324:
-------------------------------------

Awesome!! You can add the following information to the use example section. Thanks [~leftylev]
{noformat}
./hive --service metatool -updateLocation hdfs://localhost:9000 hdfs://namenode2:8020 -tablePropKey
avro.schema.url -serdePropKey avro.schema.url
Initializing HiveMetaTool..
15/04/22 14:18:42 INFO metastore.ObjectStore: ObjectStore, initialize called
15/04/22 14:18:42 INFO DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown
unknown - will be ignored
15/04/22 14:18:42 INFO DataNucleus.Persistence: Property datanucleus.cache.level2 unknown
- will be ignored
15/04/22 14:18:43 INFO metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
15/04/22 14:18:43 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema"
is tagged as "embedded-only" so does not have its own datastore table.
15/04/22 14:18:43 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder"
is tagged as "embedded-only" so does not have its own datastore table.
15/04/22 14:18:44 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema"
is tagged as "embedded-only" so does not have its own datastore table.
15/04/22 14:18:44 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder"
is tagged as "embedded-only" so does not have its own datastore table.
15/04/22 14:18:44 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0"
since the connection used is closing
15/04/22 14:18:44 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is MYSQL
15/04/22 14:18:44 INFO metastore.ObjectStore: Initialized ObjectStore
Looking for LOCATION_URI field in DBS table to update..
Successfully updated the following locations..
Updated 0 records in DBS table
Looking for LOCATION field in SDS table to update..
Successfully updated the following locations..
Updated 0 records in SDS table
Looking for value of avro.schema.url key in TABLE_PARAMS table to update..
Successfully updated the following locations..
Updated 0 records in TABLE_PARAMS table
Looking for value of avro.schema.url key in SD_PARAMS table to update..
Successfully updated the following locations..
Updated 0 records in SD_PARAMS table
Looking for value of avro.schema.url key in SERDE_PARAMS table to update..
Successfully updated the following locations..
Updated 0 records in SERDE_PARAMS table
{noformat} 

> Hive metatool should take table_param_key to allow for changes to avro serde's schema
url key
> ---------------------------------------------------------------------------------------------
>
>                 Key: HIVE-10324
>                 URL: https://issues.apache.org/jira/browse/HIVE-10324
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 1.1.0
>            Reporter: Szehon Ho
>            Assignee: Ferdinand Xu
>             Fix For: 1.2.0
>
>         Attachments: HIVE-10324.1.patch, HIVE-10324.patch, HIVE-10324.patch.WIP
>
>
> HIVE-3443 added support to change the serdeParams from 'metatool updateLocation' command.
> However, in avro it is possible to specify the schema via the tableParams:
> {noformat}
> CREATE  TABLE `testavro`(
>   `test` string COMMENT 'from deserializer')
> ROW FORMAT SERDE 
>   'org.apache.hadoop.hive.serde2.avro.AvroSerDe' 
> STORED AS INPUTFORMAT 
>   'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' 
> OUTPUTFORMAT 
>   'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
> TBLPROPERTIES (
>   'avro.schema.url'='hdfs://namenode:8020/tmp/test.avsc', 
>   'kite.compression.type'='snappy', 
>   'transient_lastDdlTime'='1427996456')
> {noformat}
> Hence for those tables the 'metatool updateLocation' will not help.
> This is necessary in case like upgrade the namenode to HA where the absolute paths have
changed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message