Mailing-List: contact reviews-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Message-Id: <201710210025.v9L0PWH2019646@ip-10-146-233-104.ec2.internal>
Date: Sat, 21 Oct 2017 00:25:32 +0000
From: "Bikramjeet Vig (Code Review)" <gerrit@cloudera.org>
To: impala-cr@cloudera.com, reviews@impala.incubator.apache.org
CC: Joe McDonnell <joemcdonnell@cloudera.com>,
        Tim Armstrong <tarmstrong@cloudera.com>
Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-5522=3AUse_tracked_memory_for_DictDecoder_and_DictEncoder=0A?=
In-Reply-To: <gerrit.1505172487000.I02a3b54f6c107d19b62ad9e1c49df94175964299@gerrit.cloudera.org>
References: <gerrit.1505172487000.I02a3b54f6c107d19b62ad9e1c49df94175964299@gerrit.cloudera.org>
Reply-To: bikramjeet.vig@cloudera.com, impala-cr@cloudera.com,
        marcelk@gmail.com, tarmstrong@cloudera.com, joemcdonnell@cloudera.com,
        reviews@impala.incubator.apache.org
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Disposition: inline
User-Agent: Gerrit/2.14.2
Content-Type: multipart/alternative; boundary="7zDFKvipqwQ="; charset=UTF-8
archived-at: Sat, 21 Oct 2017 00:25:39 -0000

--7zDFKvipqwQ=
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Bikramjeet Vig has posted comments on this change=2E ( http://gerrit=2Eclou=
dera=2Eorg:8080/8034 )

Change subject: IMPALA-5522:Use tracked memory for =
DictDecoder and DictEncoder
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E


Patch Set 11:

(2 comments)

http://gerrit=2Ecloudera=2Eo=
rg:8080/#/c/8034/10/be/src/exec/hdfs-parquet-scanner=2Eh
File be/src/exec/h=
dfs-parquet-scanner=2Eh:

http://gerrit=2Ecloudera=2Eorg:8080/#/c/8034/10/b=
e/src/exec/hdfs-parquet-scanner=2Eh@352
PS10, Line 352: return mem_tracker_=
;
> I tried that earlier but I was getting the below error for parquet tabl=
e wr
I see, that probably happens because HdfsTableSink class only has a fo=
rward declaration in hdfs-table-write=2Eh

If you would like to give this a=
nother try:

You can try including the hdfs-table-sink=2Eh in the hdfs-parq=
uet-table-writer=2Eh and it should work=2E 
OR 
you can move the implementa=
tion of dict_mem_tracker() to hdfs-parquet-table-writers=2Ecc and add the i=
nclude there instead in order to minimize the includes in the header file=
=2E


http://gerrit=2Ecloudera=2Eorg:8080/#/c/8034/10/be/src/exec/parquet-c=
olumn-readers=2Eh
File be/src/exec/parquet-column-readers=2Eh:

http://gerr=
it=2Ecloudera=2Eorg:8080/#/c/8034/10/be/src/exec/parquet-column-readers=2Eh=
@356
PS10, Line 356: if (dict_decoder_base_ !=3D nullptr) dict_decoder_base=
_->Close();
> GetDictionaryDecoder() returns dict_decoder_base_ it should b=
e the same cal
Sorry, I should have been more clearer about what I was sugg=
esting=2E What I meant was, instead of changing the method GetDictionaryDec=
oder() in parquet-column-readers=2Eh where you return dict_decoder_base_, w=
e can instead get rid of dict_decoder_base_ member var altogether and use G=
etDictionaryDecoder() because the overridden method in ScalarColumnReader w=
ould return the actual dict_decoder_ object that we can use directly and in=
 all other cases the default implementation will return a nullptr=2E


-- =

To view, visit http://gerrit=2Ecloudera=2Eorg:8080/8034
To unsubscribe, vi=
sit http://gerrit=2Ecloudera=2Eorg:8080/settings

Gerrit-Project: Impala-AS=
F
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I02a3=
b54f6c107d19b62ad9e1c49df94175964299
Gerrit-Change-Number: 8034
Gerrit-Patc=
hSet: 11
Gerrit-Owner: Pranay Singh
Gerrit-Reviewer: Bikramjeet Vig <bikram=
jeet=2Evig@cloudera=2Ecom>
Gerrit-Reviewer: Joe McDonnell <joemcdonnell@clo=
udera=2Ecom>
Gerrit-Reviewer: Pranay Singh
Gerrit-Reviewer: Tim Armstrong <=
tarmstrong@cloudera=2Ecom>
Gerrit-Comment-Date: Sat, 21 Oct 2017 00:25:32 +=
0000
Gerrit-HasComments: Yes

--7zDFKvipqwQ=--