hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Murtaza Doctor <murt...@richrelevance.com>
Subject Avro Backed Hive tables
Date Thu, 07 Mar 2013 01:43:48 GMT
Folks,

Wanted to get some help or feedback from the community on this one:

We are trying to create a table backed by avro schema:
Hive Version: 0.10

This is the syntax we are using to create the table. As you can notice the avro schema is
defined inline.
Question:-  If you look at the avro schema we have a union of two records and when we issue
the command it successfully creates the table with only the first record PageView and the
columns searchTerms & filterCategories. It completely ignored the second union record.
It is probably where hive is not supporting avro union records as part of the CREATE table
statement. Any clues on this one would help.

CREATE TABLE foo
ROW FORMAT
SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS
INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
TBLPROPERTIES ('avro.schema.literal'='{
      "type" : "record",
      "name" : "PageView",
      "fields" : [ {
        "name" : "searchTerms",
        "type" : {
          "type" : "array",
          "items" : "string"
        }
      }, {
        "name" : "filterCategories",
        "type" : {
          "type" : "array",
          "items" : "string"
        }
      } ]
    }, {
      "type" : "record",
      "name" : "CategoryView",
      "fields" : [ {
        "name" : "categoryId",
        "type" : "string"
      }, {
        "name" : "refinements",
        "type" : {
          "type" : "array",
          "items" : {
            "type" : "record",
            "name" : "Refinement",
            "fields" : [ {
              "name" : "attribute",
              "type" : "string"
            }, {
              "name" : "value",
              "type" : "string"
            } ]
          }
        },
        "default" : [ ]
      } ]
}');

Thanks,
murtaza


Mime
View raw message