hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szehon Ho (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7286) Parameterize HCatMapReduceTest for testing against all Hive storage formats
Date Thu, 10 Jul 2014 04:06:04 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14057102#comment-14057102
] 

Szehon Ho commented on HIVE-7286:
---------------------------------

OK that sounds reasonable.  So if I understand correctly, this patch will enforce test only
on serde's registered via StorageFormatDescriptor, which is trivial for serde author.  Avro
wont be tested until its registered that way, at which point there wont be a need to specify
avro-schema and hence also trivial.

bq. SerDe devs would only be required to add an entry to the table in the test with the SerDe
class and nulls in the other fields. 

Can we make a total no-op for serde author, ie the test auto-generates the parameters based
on StorageFormatDescriptor list?

> Parameterize HCatMapReduceTest for testing against all Hive storage formats
> ---------------------------------------------------------------------------
>
>                 Key: HIVE-7286
>                 URL: https://issues.apache.org/jira/browse/HIVE-7286
>             Project: Hive
>          Issue Type: Test
>          Components: HCatalog
>            Reporter: David Chen
>            Assignee: David Chen
>         Attachments: HIVE-7286.1.patch
>
>
> Currently, HCatMapReduceTest, which is extended by the following test suites:
>  * TestHCatDynamicPartitioned
>  * TestHCatNonPartitioned
>  * TestHCatPartitioned
>  * TestHCatExternalDynamicPartitioned
>  * TestHCatExternalNonPartitioned
>  * TestHCatExternalPartitioned
>  * TestHCatMutableDynamicPartitioned
>  * TestHCatMutableNonPartitioned
>  * TestHCatMutablePartitioned
> These tests run against RCFile. Currently, only TestHCatDynamicPartitioned is run against
any other storage format (ORC).
> Ideally, HCatalog should be tested against all storage formats supported by Hive. The
easiest way to accomplish this is to turn HCatMapReduceTest into a parameterized test fixture
that enumerates all Hive storage formats. Until HIVE-5976 is implemented, we would need to
manually create the mapping of SerDe to InputFormat and OutputFormat. This way, we can explicitly
keep track of which storage formats currently work with HCatalog or which ones are untested
or have test failures. The test fixture should also use Reflection to find all classes in
the classpath that implements the SerDe interface and raise a failure if any of them are not
enumerated.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message