impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Laszlo Gaal (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-6067: Enable S3 access via IAM roles for EC2 VMs
Date Tue, 05 Dec 2017 23:38:36 GMT
Laszlo Gaal has posted comments on this change. ( )

Change subject: IMPALA-6067: Enable S3 access via IAM roles for EC2 VMs

Patch Set 4:

(1 comment)
File fe/src/test/java/org/apache/impala/analysis/
PS4, Line 3000: 
> Can we restore this?  LOAD DATA INPATH doesn't get coverage in the other te
I'm facing a problem with this one.
Note, that the original test here is a negative one: it verifies that LOAD DATA INPATH rejects
an s3n: filesystem reference.
With s3n dummy credentials removed from the minicluster configuration and Hadoop3 removing
the s3n provider completely the negative test will still fail, but with a different error/exception,
and the test would not be relevant any more.

I tried to change it to a positive test using AnalyzesOk() and an s3a: reference. The problem
there was that the s3a: provider seems to insist on hitting the S3 object URI encoded in the
test. That means that a successful test step needs an existing S3 bucket and object ("file")
and valid credentials to access the file. I think it would be problematic if FE unit tests
suddenly started requiring valid S3 credentials.
I could make the S3 bucket world-readable, but it still feels awkward.

Also, the end2end tests actually test various LOAD DATA INPATH scenarios, e.g.
These tests use the FILESYSTEM_PREFIX environment variable, so they run against S3 (specifically
s3a:) when the tests are run on S3.

Would you be comfortable with disabling this test step and following it up in a separate Jira?

To view, visit
To unsubscribe, visit

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I14cd9d4453a91baad3c379aa7e4944993fca95ae
Gerrit-Change-Number: 8294
Gerrit-PatchSet: 4
Gerrit-Owner: Laszlo Gaal <>
Gerrit-Reviewer: Alex Behm <>
Gerrit-Reviewer: David Knupp <>
Gerrit-Reviewer: Jim Apple <>
Gerrit-Reviewer: Joe McDonnell <>
Gerrit-Reviewer: Lars Volker <>
Gerrit-Reviewer: Laszlo Gaal <>
Gerrit-Reviewer: Michael Brown <>
Gerrit-Reviewer: Philip Zeyliger <>
Gerrit-Reviewer: Sailesh Mukil <>
Gerrit-Reviewer: Tim Armstrong <>
Gerrit-Reviewer: Zach Amsden <>
Gerrit-Comment-Date: Tue, 05 Dec 2017 23:38:36 +0000
Gerrit-HasComments: Yes

  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message