hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-6056) The AvroSerDe gives out BadSchemaException if a partition is added to the table
Date Fri, 20 Dec 2013 00:21:07 GMT

     [ https://issues.apache.org/jira/browse/HIVE-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Xuefu Zhang updated HIVE-6056:
------------------------------

    Description: 
While creating an external table if I do not add a partition, I am able to read files using
following format: 
{code}
CREATE external TABLE event
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
LOCATION 's3n://test-event/input/2013/14/10'
TBLPROPERTIES ('avro.schema.literal' = '..some schema..');
{code}
but if I add a partition based on date
{code}
CREATE external TABLE event
PARTITIONED BY (ds STRING)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
LOCATION 's3n://test-event/input/'
TBLPROPERTIES ('avro.schema.literal' = '..some schema..');

ALTER TABLE event ADD IF NOT EXISTS PARTITION (ds = '2013_12_16') LOCATION '2013/12/16/';
{code}
I get the following exception:
{code}
java.io.IOException:org.apache.hadoop.hive.serde2.avro.BadSchemaException
{code}

  was:
While creating an external table if I do not add a partition, I am able to read files using
following format: 

CREATE external TABLE event
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
LOCATION 's3n://test-event/input/2013/14/10'
TBLPROPERTIES ('avro.schema.literal' = '..some schema..');

but if I add a partition based on date

CREATE external TABLE event
PARTITIONED BY (ds STRING)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
LOCATION 's3n://test-event/input/'
TBLPROPERTIES ('avro.schema.literal' = '..some schema..');

ALTER TABLE event ADD IF NOT EXISTS PARTITION (ds = '2013_12_16') LOCATION '2013/12/16/';

I get the following exception:
java.io.IOException:org.apache.hadoop.hive.serde2.avro.BadSchemaException


> The AvroSerDe gives out BadSchemaException if a partition is added to the table
> -------------------------------------------------------------------------------
>
>                 Key: HIVE-6056
>                 URL: https://issues.apache.org/jira/browse/HIVE-6056
>             Project: Hive
>          Issue Type: Bug
>          Components: Database/Schema
>    Affects Versions: 0.11.0
>         Environment: amazon EMR (hadoop Amazon 1.0.3), avro-1.7.5
>            Reporter: Rushil Gupta
>
> While creating an external table if I do not add a partition, I am able to read files
using following format: 
> {code}
> CREATE external TABLE event
> ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
> STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
> OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
> LOCATION 's3n://test-event/input/2013/14/10'
> TBLPROPERTIES ('avro.schema.literal' = '..some schema..');
> {code}
> but if I add a partition based on date
> {code}
> CREATE external TABLE event
> PARTITIONED BY (ds STRING)
> ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
> STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
> OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
> LOCATION 's3n://test-event/input/'
> TBLPROPERTIES ('avro.schema.literal' = '..some schema..');
> ALTER TABLE event ADD IF NOT EXISTS PARTITION (ds = '2013_12_16') LOCATION '2013/12/16/';
> {code}
> I get the following exception:
> {code}
> java.io.IOException:org.apache.hadoop.hive.serde2.avro.BadSchemaException
> {code}



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Mime
View raw message