impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Quanlong Huang (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-9859: Full ACID Milestone 4: Part 1 Reading modified tables (primitive types)
Date Wed, 01 Jul 2020 06:57:52 GMT
Quanlong Huang has posted comments on this change. (

Change subject: IMPALA-9859: Full ACID Milestone 4: Part 1 Reading modified tables (primitive

Patch Set 9:

File fe/src/main/java/org/apache/impala/planner/
PS9, Line 45: import org.apache.impala.analysis.Path.PathType;
nit: keep the import list sorted in groups (usually the IDE will do this for you).
PS9, Line 1418:       if (addAcidSlotsIfNeeded(analyzer, hdfsTblRef, partitions)) {
nit: what about merging this if-statement with its outer scope so they are

 if (isPartitionKeyScan && queryOpts.optimize_partition_key_scans) {
 } else if (addAcidSlotsIfNeeded(analyzer, hdfsTblRef, partitions)) {
 } else {
PS9, Line 1518:     // Let's separate insert delta File Descriptors from delete delta FDs.
I think we should separate the file descriptors in catalogd after loading them from HDFS instead
of doing it here for each query. We can introduce two fileds in HdfsPartition: encodedInsertDeltaFileDescriptors_
and encodedDeleteDeltaFileDescriptors_ (and related fields in THdfsPartition and TPartialPartitionInfo).
If a partition contains delete deltas, we separate them by setting these two fields and leaving
encodedFileDescriptors_ null. We can also introduce two methods for FeFsPartition: genInsertDeltaPartition()
and genDeleteDeltaPartition() using HdfsPartition.Builder in this way:

 public HdfsPartition genInsertDeltaPartition() {
    return new HdfsPartition.Builder(this)

With this we don't need to remove the "final" marker of encodedFileDescriptors_ and add back
the setFileDescriptor() method, which violates our goal to make HdfsPartition immutable. The
setFileDescriptor() method may encourage future developers to modify HdfsPartitions in-place
in catalogd, which will break IMPALA-7533.

To view, visit
To unsubscribe, visit

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I15c8feabf40be1658f3dd46883f5a1b2aa5d0659
Gerrit-Change-Number: 16082
Gerrit-PatchSet: 9
Gerrit-Owner: Zoltan Borok-Nagy <>
Gerrit-Reviewer: Aman Sinha <>
Gerrit-Reviewer: Impala Public Jenkins <>
Gerrit-Reviewer: Quanlong Huang <>
Gerrit-Reviewer: Tim Armstrong <>
Gerrit-Reviewer: Zoltan Borok-Nagy <>
Gerrit-Comment-Date: Wed, 01 Jul 2020 06:57:52 +0000
Gerrit-HasComments: Yes

  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message