impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-4993: extend dictionary filtering to collections
Date Tue, 09 Jan 2018 19:23:40 GMT
Tim Armstrong has posted comments on this change. ( )

Change subject: IMPALA-4993: extend dictionary filtering to collections

Patch Set 6:


Took a detailed look at the backend
File be/src/exec/hdfs-parquet-scanner.h:
PS6, Line 462:   std::unordered_map<const TupleDescriptor*, std::unique_ptr<ScopedBuffer>>
It would make more sense at this point to use a MemPool instead of a proliferation of ScopedBuffers
in dict_filter_tuple_map_ and min_max_tuple_buffer_ - it's the canonical way to make multiple
small allocations that are freed at the same time. It would have made sense even before this

level_cache_pool_ already has the right lifetime (freed in close) so we could just rename
that to something more generic  that reflects the lifetime. E.g. we call something similar
a "perm_pool" elsewhere because its contents have the same lifetime as the owning object.
PS6, Line 649:   /// Gets the TupleDescriptor of slot_desc.
Mention that 'slot_desc' can belong to the top-level tuple or a tuple nested in a collection?
File be/src/exec/
PS6, Line 1620: 
thanks :)
PS6, Line 771:   if (!col_reader->IsCollectionReader()) {
nit: I think this would be easier to follow if we reversed the branches and removed the negation.
PS6, Line 815:   for (auto* col_reader : column_readers_) {
nit: could fit on one line now
PS6, Line 1657:   if (column_readers.empty()) return Status::OK();
Is the early exit necessary for correctness? Might be worth mentioning if it is.

Otherwise, I don't think it matters for performance so my bias is towards leaving it out.

To view, visit
To unsubscribe, visit

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If3a2abcfc3d0f7d18756816659fed77ce12668dd
Gerrit-Change-Number: 8775
Gerrit-PatchSet: 6
Gerrit-Owner: Vuk Ercegovac <>
Gerrit-Reviewer: Alex Behm <>
Gerrit-Reviewer: Lars Volker <>
Gerrit-Reviewer: Tim Armstrong <>
Gerrit-Reviewer: Vuk Ercegovac <>
Gerrit-Comment-Date: Tue, 09 Jan 2018 19:23:40 +0000
Gerrit-HasComments: Yes

  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message