Mailing-List: contact reviews-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Message-Id: <201705241650.v4OGoxUA032185@ip-10-146-233-104.ec2.internal>
Date: Wed, 24 May 2017 16:50:59 +0000
From: "Tim Armstrong (Code Review)" <gerrit@cloudera.org>
To: impala-cr@cloudera.com, reviews@impala.incubator.apache.org
CC: anujphadke <aphadke@cloudera.com>, Jim Apple <jbapple-impala@apache.org>,
        Mostafa Mokhtar <mmokhtar@cloudera.com>,
        Marcel Kornacker <marcel@cloudera.com>,
        Alex Behm <alex.behm@cloudera.com>
Reply-To: tarmstrong@cloudera.com
Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-5347=3A_Parquet_scanner_microoptimizations=0A?=
In-Reply-To: <gerrit.1495470995000.I49ec523a65542fdbabd53fbcc4a8901d769e5cd5@gerrit.cloudera.org>
References: <gerrit.1495470995000.I49ec523a65542fdbabd53fbcc4a8901d769e5cd5@gerrit.cloudera.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Content-Disposition: inline
User-Agent: Gerrit/2.12.7
archived-at: Wed, 24 May 2017 16:51:06 -0000

Tim Armstrong has posted comments on this change.

Change subject: IMPALA-5347: Parquet scanner microoptimizations
......................................................................


Patch Set 8:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/6950/8/be/src/exec/hdfs-parquet-scanner.cc
File be/src/exec/hdfs-parquet-scanner.cc:

Line 991:       InitTuple(template_tuple_, scratch_batch_->GetTuple(i));
> does the extra branch in InitTuple not matter because it's always perfectly
Good point. I think the bottleneck there is probably more the branching inside memcpy(), since byte_size isn't a compile-time constant, but we might as well make this more efficient.

I moved the logic to an InitTupleBuffer() function in HdfsScanner and use a more optimised implementation for all three cases.

I did some basic profiling of scans of a partitioned table with perf top and it seems like most of the time is spent inside memcpy() both before and after the change.


-- 
To view, visit http://gerrit.cloudera.org:8080/6950
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I49ec523a65542fdbabd53fbcc4a8901d769e5cd5
Gerrit-PatchSet: 8
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbapple-impala@apache.org>
Gerrit-Reviewer: Marcel Kornacker <marcel@cloudera.com>
Gerrit-Reviewer: Mostafa Mokhtar <mmokhtar@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Reviewer: anujphadke <aphadke@cloudera.com>
Gerrit-HasComments: Yes