hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szehon Ho (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7286) Parameterize HCatMapReduceTest for testing against all Hive storage formats
Date Tue, 15 Jul 2014 20:02:05 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062562#comment-14062562
] 

Szehon Ho commented on HIVE-7286:
---------------------------------

Hi David, can we get rid of StorageCustomHandler once HIVE-6806 is committed?  Would it make
sense to commit that first, so this is cleaner?

Also just my two cents, I'm still not in favor of the test inspecting the classpath and failing
if a certain serde hasn't been either registered by StorageFormatDescriptor or excluded by
the test.  Like what if you wrote a test serde, then you need to go to this test and put it
on the exclude list otherwise it will fail.  Or what if you have a dirty classpath, it still
seems a bit flaky to me.  Can we get rid of that part?  Again I'm ok with generating parameters
for the StorageFormatDescriptor serdes and even the 'additional' serdes

> Parameterize HCatMapReduceTest for testing against all Hive storage formats
> ---------------------------------------------------------------------------
>
>                 Key: HIVE-7286
>                 URL: https://issues.apache.org/jira/browse/HIVE-7286
>             Project: Hive
>          Issue Type: Test
>          Components: HCatalog
>            Reporter: David Chen
>            Assignee: David Chen
>         Attachments: HIVE-7286.1.patch, HIVE-7286.2.patch, HIVE-7286.3.patch, HIVE-7286.4.patch
>
>
> Currently, HCatMapReduceTest, which is extended by the following test suites:
>  * TestHCatDynamicPartitioned
>  * TestHCatNonPartitioned
>  * TestHCatPartitioned
>  * TestHCatExternalDynamicPartitioned
>  * TestHCatExternalNonPartitioned
>  * TestHCatExternalPartitioned
>  * TestHCatMutableDynamicPartitioned
>  * TestHCatMutableNonPartitioned
>  * TestHCatMutablePartitioned
> These tests run against RCFile. Currently, only TestHCatDynamicPartitioned is run against
any other storage format (ORC).
> Ideally, HCatalog should be tested against all storage formats supported by Hive. The
easiest way to accomplish this is to turn HCatMapReduceTest into a parameterized test fixture
that enumerates all Hive storage formats. Until HIVE-5976 is implemented, we would need to
manually create the mapping of SerDe to InputFormat and OutputFormat. This way, we can explicitly
keep track of which storage formats currently work with HCatalog or which ones are untested
or have test failures. The test fixture should also use Reflection to find all classes in
the classpath that implements the SerDe interface and raise a failure if any of them are not
enumerated.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message