kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From t...@apache.org
Subject [3/3] kudu git commit: [tests] fix flakiness in catalog_manager_tsk-itest
Date Sat, 26 Aug 2017 01:28:53 GMT
[tests] fix flakiness in catalog_manager_tsk-itest

After recent updates the catalog_manager_tsk-itest became unstable
in TSAN configuration.  The failure scenario looks like the following:

  * 3 masters are running with --raft_enable_pre_election=false flag
    and short raft heartbeat interval.

  * The test client sends CreateTable request. The server side completes
    the request, but once the storm of master re-elections starts,
    the client cannot get the authoritative answer on its
    IsCreateTableDone requests because master leadership changes
    every 40-100ms.  As as result, after retrying IsCreateTableDone
    request many times, the client times out on its CreateTable request.

Making the raft heartbeat interval higher helps keeping the master
leadership intervals longer, so the client has better chance of
getting authoritative response from the masters.

Change-Id: I3731fb560923d82d2fb64ac7b829080f99a0681e
Reviewed-on: http://gerrit.cloudera.org:8080/7823
Tested-by: Alexey Serbin <aserbin@cloudera.com>
Reviewed-by: Todd Lipcon <todd@apache.org>

Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/faa0b14e
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/faa0b14e
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/faa0b14e

Branch: refs/heads/master
Commit: faa0b14effb6e15f9989d686e5a1f8e1040a1dd6
Parents: a07d01e
Author: Alexey Serbin <aserbin@cloudera.com>
Authored: Thu Aug 24 18:19:41 2017 -0700
Committer: Todd Lipcon <todd@apache.org>
Committed: Sat Aug 26 01:28:26 2017 +0000

 src/kudu/integration-tests/catalog_manager_tsk-itest.cc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/kudu/integration-tests/catalog_manager_tsk-itest.cc b/src/kudu/integration-tests/catalog_manager_tsk-itest.cc
index acdeba9..e573097 100644
--- a/src/kudu/integration-tests/catalog_manager_tsk-itest.cc
+++ b/src/kudu/integration-tests/catalog_manager_tsk-itest.cc
@@ -61,7 +61,7 @@ class CatalogManagerTskITest : public KuduTest {
       : num_masters_(3),
-        hb_interval_ms_(32),
+        hb_interval_ms_(64),

View raw message