impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Knupp (JIRA)" <j...@apache.org>
Subject [jira] [Created] (IMPALA-5330) Impala tests never use or set a secondary FS, so TestMultipleFilesystems is always skipped
Date Tue, 16 May 2017 00:19:04 GMT
David Knupp created IMPALA-5330:
-----------------------------------

             Summary: Impala tests never use or set a secondary FS, so TestMultipleFilesystems
is always skipped
                 Key: IMPALA-5330
                 URL: https://issues.apache.org/jira/browse/IMPALA-5330
             Project: IMPALA
          Issue Type: Bug
          Components: Infrastructure
            Reporter: David Knupp
            Priority: Critical
             Fix For: Impala 2.9.0


This issue came up while investigating how can set up Impala + ADLS testing on remote clusters.
Since testing Impala with ADLS is analogous to testing Impala with s3, I used that testing
as a starting point.

When we change the TARGET_FILESYSTEM to 's3', the end result is to overwrite the defaultFS
setting in core-site.xml to the s3 location. There's not a point (or doesn't appear to be
a point) where we ever treat s3 as a true secondary filesystem.

Further more, if we look at our test suite that is supposed to check multiple filesystems
simultaneously, we can see there's a pytest SkipIf decorator set to skip the test if a secondary
FS is not set.

{noformat}
@SkipIf.no_secondary_fs
class TestMultipleFilesystems(ImpalaTestSuite)
{noformat}

>From what I can see by going through old Jenkins console logs, this test never gets executed.
It's skipped during exhaustive runs with HDFS as the default, and is also skipped during s3
and local FS test runs as well.

{noformat}
04:27:57 query_test/test_multiple_filesystems.py::TestMultipleFilesystems::test_multiple_filesystems[exec_option:
{'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0, 'batch_size':
0, 'num_nodes': 0} | table_format: text/none] SKIPPED
{noformat}




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message