impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Laszlo Gaal (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-6067: Enable s3 access via IAM roles for EC2 VMs
Date Wed, 08 Nov 2017 15:24:18 GMT
Laszlo Gaal has posted comments on this change. ( )

Change subject: IMPALA-6067: Enable s3 access via IAM roles for EC2 VMs

Patch Set 1:

File bin/
             : #export AWS_ACCESS_KEY_ID="${AWS_ACCESS_KEY_ID-}"
> Self-inflicted review finding: remove commented-out lines.
PS1, Line 256: if [[ -z ${AWS_ACCESS_KEY_ID-} ]]; then
> (Bash hackery)
Thanks for the tip, Philip, I ended up using it.
PS1, Line 268: test
> tests
PS1, Line 276: curl -sf --connect-timeout 1 --max-time 5
> Are these curl options common across a wide variety of OSs? If they are new
Yes. I have tested the curl options on all the platforms on which the packaging build runs.
Curl understands these options on all these platforms.
PS1, Line 276:
> Is there a AWS reference page you can add as a comment so I can read up mor
PS1, Line 276:
I have put the URL to the Amazon doc page where this mechanism is described.
File fe/src/test/java/org/apache/impala/analysis/
PS1, Line 3000:       AnalysisError(String.format("load data inpath '%s' %s into table " +
              :           "tpch.lineitem", "s3a://bucket/test-warehouse/test.out", overwrite),
              :           "INPATH location 's3a://bucket/test-warehouse/test.out' must point
to an " +
              :           "HDFS, S3A or ADL filesystem.");
> Impala cannot load data from s3n. I think this test is intended to verify o
That's right; I was pattern-matching too eagerly. Thanks for catching this.
I have in fact reverted all the s3n->s3a changes in the FE tests; looks like they don't
in fact touch S3 and they are OK with some existing but obviously fake credentials in core-site.xml.
File testdata/cluster/node_templates/common/etc/hadoop/conf/core-site.xml.tmpl:
PS1, Line 61: <!-- BEGIN s3 security settings -->
> Maybe have a comment explaining testdata/cluster/admin needs this (and the 

referred the Amazon doc site as well.

To view, visit
To unsubscribe, visit

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I14cd9d4453a91baad3c379aa7e4944993fca95ae
Gerrit-Change-Number: 8294
Gerrit-PatchSet: 1
Gerrit-Owner: Laszlo Gaal <>
Gerrit-Reviewer: Alex Behm <>
Gerrit-Reviewer: David Knupp <>
Gerrit-Reviewer: Jim Apple <>
Gerrit-Reviewer: Joe McDonnell <>
Gerrit-Reviewer: Lars Volker <>
Gerrit-Reviewer: Laszlo Gaal <>
Gerrit-Reviewer: Michael Brown <>
Gerrit-Reviewer: Philip Zeyliger <>
Gerrit-Reviewer: Sailesh Mukil <>
Gerrit-Comment-Date: Wed, 08 Nov 2017 15:24:18 +0000
Gerrit-HasComments: Yes

  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message