atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-409) Atlas will not import avro tables with schema read from a file
Date Tue, 02 Feb 2016 07:45:40 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15127840#comment-15127840
] 

Hemanth Yamijala commented on ATLAS-409:
----------------------------------------

[~dossett], Shwetha and I were looking at the patch for ATLAS-443 which is trying to add a
test case for the fix in this JIRA. We found that the  test does not fail even without the
patch for ATLAS-409. The JSON that is being sent as part of test contains the columns field
(cols) and hence does not reproduce the exact scenario. I will attach the JSON that shows
up in the test application log to this JIRA for your review. We suspect you got the error
posted earlier on this JIRA either because you are using a different version of Atlas or Hive.
Could you please provide some more details to help us understand the case when this shows
up?

To be clear, we are seeing the behavior of columns showing up for the Avro file with Atlas
master branch and Hive 1.2.1.



> Atlas will not import avro tables with schema read from a file
> --------------------------------------------------------------
>
>                 Key: ATLAS-409
>                 URL: https://issues.apache.org/jira/browse/ATLAS-409
>             Project: Atlas
>          Issue Type: Bug
>    Affects Versions: 0.6-incubating
>            Reporter: Aaron Dossett
>            Assignee: Aaron Dossett
>             Fix For: trunk
>
>         Attachments: ATLAS-409-2.patch, ATLAS-409-3.patch, ATLAS-409.patch, example.ddl,
foo.avsc
>
>
> Atlas won't import a Hive table with no columns (see below for an example of a valid
hive table with no explicit columns).  This is because the Atlas Hive Storage Descriptor class
REQUIRES columns, but the Hive Table class allows them to be OPTIONAL.
> {code}
> CREATE TABLE example
>   ROW FORMAT SERDE
>   'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
>   STORED AS INPUTFORMAT
>   'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
>   OUTPUTFORMAT
>   'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
>   TBLPROPERTIES (
>     'avro.schema.url'='file:///path/to/the/schema/test_serializer.avsc');
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message