impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Ho (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-2831: Bound the number of scanner threads per scan node.
Date Tue, 30 Aug 2016 23:22:16 GMT
Michael Ho has posted comments on this change.

Change subject: IMPALA-2831: Bound the number of scanner threads per scan node.
......................................................................


Patch Set 1:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/4174/1/be/src/exec/hdfs-scan-node.cc
File be/src/exec/hdfs-scan-node.cc:

Line 262:     RETURN_IF_ERROR(HdfsParquetScanner::IssueInitialRanges(this,
> Is the order important? Would be helpful to have a comment here if there is
Yes, mostly for slight preference for parquet files. This mostly makes up for the difference
in behavior due to the change in line 906. Previously, we would have spun up those scanner
threads (unintentionally) when calling IssueInitalRanges() for Text file format. We also start
issuing the initial IO ranges slightly earlier now for parquet files.


Line 740:     runtime_state_->resource_pool()->set_max_quota(CpuInfo::num_cores() +
1);
> Nm, you are already doing that. It seems weird that the scanner threads quo
Yes, I agree that we probably should have the + 1 for the query option too.


-- 
To view, visit http://gerrit.cloudera.org:8080/4174
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I191988ad18d6b4caf892fc967258823edcf9681f
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-HasComments: Yes

Mime
View raw message