impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikramjeet Vig (Code Review)" <>
Subject [Impala-CR](cdh5-trunk) IMPALA-1683: Allow REFRESH on a single partition
Date Fri, 29 Jul 2016 06:31:03 GMT
Hello David Knupp, Henry Robinson, Dimitris Tsirogiannis, Alex Behm,

I'd like you to reexamine a change.  Please visit

to look at the new patch set (#13).

Change subject: IMPALA-1683: Allow REFRESH on a single partition

IMPALA-1683: Allow REFRESH on a single partition

Currently the only way to refresh metadata for a partition was to refresh
the whole table. This is a relatively time consuming process especially if
there are many partitions and only one is to be refreshed.
This patch allows the client to REFRESH on a single partition by using the
following syntax:
REFRESH [database_name.]table_name PARTITION (partition_spec)

Added parsing and authorization tests in and respectively. A new test file
"" was added for testing functionality.

For a table with 10000 partitions and 1 file per partition

                     execResetMetadata()       Total Execution Time

Refresh Table              3795 ms                   4630 ms

Refersh Partition            42 ms                    680 ms

We see that the time to refresh improves by a factor of 90x but due to
significant overhead of about 640ms in this case the effective improvement
is about 7x. As the size of the table and number of partitions increase,
this improvement would be more significant.

Change-Id: I4bf5dd630f2e175be5800187bbf9337c91f82bd8
M common/thrift/CatalogService.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/com/cloudera/impala/analysis/
M fe/src/main/java/com/cloudera/impala/analysis/
M fe/src/main/java/com/cloudera/impala/catalog/
M fe/src/main/java/com/cloudera/impala/catalog/
M fe/src/main/java/com/cloudera/impala/service/
M fe/src/test/java/com/cloudera/impala/analysis/
M fe/src/test/java/com/cloudera/impala/analysis/
M fe/src/test/java/com/cloudera/impala/analysis/
A tests/metadata/
11 files changed, 406 insertions(+), 7 deletions(-)

  git pull ssh:// refs/changes/85/3385/13
To view, visit
To unsubscribe, visit

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4bf5dd630f2e175be5800187bbf9337c91f82bd8
Gerrit-PatchSet: 13
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Bikramjeet Vig <>
Gerrit-Reviewer: Alex Behm <>
Gerrit-Reviewer: Bharath Vissapragada <>
Gerrit-Reviewer: Bikramjeet Vig <>
Gerrit-Reviewer: David Knupp <>
Gerrit-Reviewer: Dimitris Tsirogiannis <>
Gerrit-Reviewer: Henry Robinson <>
Gerrit-Reviewer: Michael Brown <>

View raw message