impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikramjeet Vig (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5522:Use tracked memory for DictDecoder and DictEncoder
Date Sat, 21 Oct 2017 00:25:32 GMT
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/8034 )

Change subject: IMPALA-5522:Use tracked memory for DictDecoder and DictEncoder
......................................................................


Patch Set 11:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/8034/10/be/src/exec/hdfs-parquet-scanner.h
File be/src/exec/hdfs-parquet-scanner.h:

http://gerrit.cloudera.org:8080/#/c/8034/10/be/src/exec/hdfs-parquet-scanner.h@352
PS10, Line 352: return mem_tracker_;
> I tried that earlier but I was getting the below error for parquet table wr
I see, that probably happens because HdfsTableSink class only has a forward declaration in
hdfs-table-write.h

If you would like to give this another try:

You can try including the hdfs-table-sink.h in the hdfs-parquet-table-writer.h and it should
work. 
OR 
you can move the implementation of dict_mem_tracker() to hdfs-parquet-table-writers.cc and
add the include there instead in order to minimize the includes in the header file.


http://gerrit.cloudera.org:8080/#/c/8034/10/be/src/exec/parquet-column-readers.h
File be/src/exec/parquet-column-readers.h:

http://gerrit.cloudera.org:8080/#/c/8034/10/be/src/exec/parquet-column-readers.h@356
PS10, Line 356: if (dict_decoder_base_ != nullptr) dict_decoder_base_->Close();
> GetDictionaryDecoder() returns dict_decoder_base_ it should be the same cal
Sorry, I should have been more clearer about what I was suggesting. What I meant was, instead
of changing the method GetDictionaryDecoder() in parquet-column-readers.h where you return
dict_decoder_base_, we can instead get rid of dict_decoder_base_ member var altogether and
use GetDictionaryDecoder() because the overridden method in ScalarColumnReader would return
the actual dict_decoder_ object that we can use directly and in all other cases the default
implementation will return a nullptr.



-- 
To view, visit http://gerrit.cloudera.org:8080/8034
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02a3b54f6c107d19b62ad9e1c49df94175964299
Gerrit-Change-Number: 8034
Gerrit-PatchSet: 11
Gerrit-Owner: Pranay Singh
Gerrit-Reviewer: Bikramjeet Vig <bikramjeet.vig@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <joemcdonnell@cloudera.com>
Gerrit-Reviewer: Pranay Singh
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Comment-Date: Sat, 21 Oct 2017 00:25:32 +0000
Gerrit-HasComments: Yes

Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message