impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dimitris Tsirogiannis (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5333: Add support for Impala to work with ADLS
Date Fri, 19 May 2017 17:38:00 GMT
Dimitris Tsirogiannis has posted comments on this change.

Change subject: IMPALA-5333: Add support for Impala to work with ADLS
......................................................................


Patch Set 3:

I have a few high level questions about this patch. This patch treats S3 and ADL the same
way but after looking at the HDFS classes and the ADL API I see some differences between ADL
and S3 API. For instance, I don't see a way in ADL to recursively enumerate all entries under
a particular directory (something that exists in S3). Our code today for S3 sort of relies
on that property so I am wondering if our code works as expected or if I am missing something
(quite possible). Also, the AdlFileSystem class has functions for returning the block locations,
which means that we don't have to call the synthesize metadata calls as we do for S3. In the
long run, HDFS may expose replica locations from the ADL local tiers (e.g. Cosmos), but I
guess this is not relevant today. Thoughts?

-- 
To view, visit http://gerrit.cloudera.org:8080/6910
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ic56b9988b32a330443f24c44f9cb2c80842f7542
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Sailesh Mukil <sailesh@cloudera.com>
Gerrit-Reviewer: Attila Jeges <attilaj@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: David Knupp <dknupp@cloudera.com>
Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sailesh@cloudera.com>
Gerrit-HasComments: No

Mime
View raw message