incubator-hcatalog-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas M Nair (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HCATALOG-49) Support Avro Data File Format in HCatalog
Date Tue, 12 Jul 2011 21:51:00 GMT

    [ https://issues.apache.org/jira/browse/HCATALOG-49?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13064168#comment-13064168
] 

Thejas M Nair commented on HCATALOG-49:
---------------------------------------

Comments HCATALOG-49.patch 

- AvroInputStorageDriver.getTypedObj and AvroOutputStorageDriver.getTypedObj should be called
recursively for map and list values. Complex type is supported in the schema validation done
in TypeConverter .

- AvroInputStorageDriver.convertToHCatRecord converts char field names to lower case before
looking up in avro schema, but TypeConverter.check doesn't. Does avro Schema.getField do case
sensitive comparison of field names ?

- AvroOutputStorageDriver.convertValue - it will bit more efficient to loop on the position
(for (int i=0; i < outputSchema., that way the column name does not have to be looked up
in outputSchema (ie HCatRecord.get(i) can be used instead of HCatRecord.get(name, schema)
)

- TestAvroInputStorageDriver - I think it will be useful to have some test cases for the case
where only some of the fields are requested, and the case when some of the fields are partition
keys.

(FYI, I am not a committer on HCatalog.)


> Support Avro Data File Format in HCatalog
> -----------------------------------------
>
>                 Key: HCATALOG-49
>                 URL: https://issues.apache.org/jira/browse/HCATALOG-49
>             Project: HCatalog
>          Issue Type: New Feature
>            Reporter: Tom White
>         Attachments: HCATALOG-49.patch
>
>
> Add input and output drivers for Avro.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message