impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Behm (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-4172/IMPALA-3653: Improvements to block metadata loading
Date Wed, 30 Nov 2016 05:14:40 GMT
Alex Behm has posted comments on this change.

Change subject: IMPALA-4172/IMPALA-3653: Improvements to block metadata loading

Patch Set 12: Code-Review+1


I'm pretty happy with the changes. Let's let Dimitris take a look.
Commit Message:

Line 15: We loop throuh each and every file in the table/partition directories

Line 17: This results in large no. of RPC calls to namenode, especially with
no. -> number

to the NameNode

especially for large tables

Line 35: 
mention the behavior change of REFRESH
File fe/src/main/java/org/apache/impala/catalog/

Line 31:  * - To maintain consistent mapping across all the table instances so that the disk
... a consistent mapping ... so that the assignment of scan ranges to I/O threads is balanced
and consistent for all scans on the same host.
File fe/src/main/java/org/apache/impala/catalog/

Line 288:    * Queries the filesystem to load the file block metadata (e.g. DFS blocks) for
Suggest rephrasing:

Drops and re-loads the block metadata for all partitions in 'partsByPath' whose location is
under the given 'dirPath'. It involves the following steps:

Line 296:    *   and enumerate all its blocks and their corresponding hosts and disk IDs.
remove part about disk ids, I think that's the next point

Line 778:     LOG.debug("partsByPath size: " + partsByPath.size());
check log lvl

To view, visit
To unsubscribe, visit

Gerrit-MessageType: comment
Gerrit-Change-Id: Ie127658172e6e70dae441374530674a4ac9d5d26
Gerrit-PatchSet: 12
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Bharath Vissapragada <>
Gerrit-Reviewer: Alex Behm <>
Gerrit-Reviewer: Bharath Vissapragada <>
Gerrit-Reviewer: Mostafa Mokhtar <>
Gerrit-HasComments: Yes

View raw message