impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Internal Jenkins (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-4633: Change broken gflag default for Kudu client mem
Date Thu, 15 Dec 2016 04:39:22 GMT
Internal Jenkins has submitted this change and it was merged.

Change subject: IMPALA-4633: Change broken gflag default for Kudu client mem
......................................................................


IMPALA-4633: Change broken gflag default for Kudu client mem

We discovered that the current Kudu client defaults in the
KuduTableSink are causing a large number of queries to
timeout, failing the query. The current default value of the
'mutation buffer size' is 100MB which results in higher
write throughput than Kudu can currently handle on large
clusters.  By decreasing the value of this flag, more RPCs
will be sent for the same amount of data, i.e. throttling
the load on Kudu. We found tests to be more successful on
200 nodes with a 10MB buffer size than the previous 100MB
value where most queries couldn't complete due to timeouts.
These queries were not timing out with the 10MB value. This
appears to work well on 9 node stress tests as well.

Change-Id: I0b3544f9a93c82e347f6e97540d6b561c30d09fd
Reviewed-on: http://gerrit.cloudera.org:8080/5503
Reviewed-by: Dan Hecht <dhecht@cloudera.com>
Tested-by: Internal Jenkins
---
M be/src/exec/kudu-table-sink.cc
M be/src/exec/kudu-table-sink.h
2 files changed, 5 insertions(+), 4 deletions(-)

Approvals:
  Internal Jenkins: Verified
  Dan Hecht: Looks good to me, approved



-- 
To view, visit http://gerrit.cloudera.org:8080/5503
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I0b3544f9a93c82e347f6e97540d6b561c30d09fd
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Matthew Jacobs <mj@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Matthew Jacobs <mj@cloudera.com>

Mime
View raw message