impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sailesh Mukil (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) Increase default fs.s3a.connection.maximum
Date Thu, 05 May 2016 18:54:02 GMT
Sailesh Mukil has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/2965

Change subject: Increase default fs.s3a.connection.maximum
......................................................................

Increase default fs.s3a.connection.maximum

Through emprical analysis, it was determined that setting the maximum
number of connections to S3 as 1500 was optimal for functionality and
performance. The hadoop set default of 15 connections could lead us to
have deadlocks as our parquet scanner requires that we have multiple
concurrent open connections proportional to the number of columns that
we are scanning.

Setting it to this high a value does not seem to have any negative
implications.

Change-Id: Ide6f1326d5155b2e5f4da3a3f23df3f3d40c5a8d
---
M testdata/cluster/node_templates/common/etc/hadoop/conf/core-site.xml.tmpl
1 file changed, 5 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/65/2965/1
-- 
To view, visit http://gerrit.cloudera.org:8080/2965
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ide6f1326d5155b2e5f4da3a3f23df3f3d40c5a8d
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Sailesh Mukil <sailesh@cloudera.com>

Mime
View raw message