impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sailesh Mukil (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-2840: Don't store table location in partition location
Date Tue, 08 Mar 2016 06:40:27 GMT
Sailesh Mukil has posted comments on this change.

Change subject: IMPALA-2840: Don't store table location in partition location
......................................................................


Patch Set 2:

> > Wouldn't this be a problem for tables which have different
 > > partitions on different filesystems?
 > 
 > I'm not sure how such a problem would arise with this approach. Can
 > you elaborate?

The patch I'm working on to support INSERTs on S3 relies on the absolute path of partitions
to figure out if a partition is on HDFS or on S3 (or any filesystem for that matter).

But looking at this patch more, if the table is created in HDFS and one of its partitions
is on S3 (which is a completely different location), the S3 partition is not relative to the
base dir. So that partition will have its complete absolute path?

Even if that's the case it will still break the assumption that we always have the absolute
path, in my patch. But once this gets in, I'm sure we can find a way to work with this new
approach.

-- 
To view, visit http://gerrit.cloudera.org:8080/2355
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I8c67b6ce0f83de2f5277a528a9ce67e47d638adb
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Jim Apple <jbapple@cloudera.com>
Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbapple@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <marcel@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sailesh@cloudera.com>
Gerrit-HasComments: No

Mime
View raw message