kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mpe...@apache.org
Subject [1/3] kudu git commit: Fix flaky raft_consensus_failure_detector-imc-itest
Date Sat, 02 Dec 2017 05:31:56 GMT
Repository: kudu
Updated Branches:
  refs/heads/branch-1.6.x d43404166 -> 0cd54e0e8


Fix flaky raft_consensus_failure_detector-imc-itest

When I wrote this test, I accidentally wrote it as if failure detection
was disabled, but of course failure detection cannot be disabled in this
failure detection test!

This was about 15% flaky before this patch. After the patch, 200/200
runs passed successfully.

Change-Id: I87dd39929d71ded5660508a383715883e00ccf8d
Reviewed-on: http://gerrit.cloudera.org:8080/8721
Reviewed-by: Alexey Serbin <aserbin@cloudera.com>
Tested-by: Kudu Jenkins
Reviewed-on: http://gerrit.cloudera.org:8080/8738
Tested-by: Mike Percy <mpercy@apache.org>
Reviewed-by: Mike Percy <mpercy@apache.org>


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/10bca9d9
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/10bca9d9
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/10bca9d9

Branch: refs/heads/branch-1.6.x
Commit: 10bca9d9d215d003d6faf7993f06a861d3d45a1b
Parents: d434041
Author: Mike Percy <mpercy@apache.org>
Authored: Fri Dec 1 17:49:20 2017 -0800
Committer: Mike Percy <mpercy@apache.org>
Committed: Sat Dec 2 05:31:17 2017 +0000

----------------------------------------------------------------------
 ...raft_consensus_failure_detector-imc-itest.cc | 26 +++++++++-----------
 1 file changed, 12 insertions(+), 14 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kudu/blob/10bca9d9/src/kudu/integration-tests/raft_consensus_failure_detector-imc-itest.cc
----------------------------------------------------------------------
diff --git a/src/kudu/integration-tests/raft_consensus_failure_detector-imc-itest.cc b/src/kudu/integration-tests/raft_consensus_failure_detector-imc-itest.cc
index 32acff3..773b3c0 100644
--- a/src/kudu/integration-tests/raft_consensus_failure_detector-imc-itest.cc
+++ b/src/kudu/integration-tests/raft_consensus_failure_detector-imc-itest.cc
@@ -69,6 +69,11 @@ TEST_F(RaftConsensusFailureDetectorIMCTest, TestFailureDetectorActivation)
{
   NO_FATALS(StartCluster(/*num_tablet_servers=*/ kNumReplicas + 1));
   TestWorkload workload(cluster_.get());
   workload.Setup();
+  workload.Start();
+  while (workload.batches_completed() < 10) {
+    SleepFor(MonoDelta::FromMilliseconds(10));
+  }
+  workload.StopAndJoin();
 
   // Identify the tablet id and locate the replicas.
   string tablet_id;
@@ -95,17 +100,6 @@ TEST_F(RaftConsensusFailureDetectorIMCTest, TestFailureDetectorActivation)
{
     ASSERT_FALSE(tablet_id.empty());
   });
 
-  // Wait until tablets are running.
-  auto range = replica_map_.equal_range(tablet_id);
-  for (auto it = range.first; it != range.second; ++it) {
-    auto ts = it->second;
-    ASSERT_OK(WaitUntilTabletRunning(ts, tablet_id, kTimeout));
-  }
-  // Elect a leader.
-  string leader_uuid = range.first->second->uuid();
-  ASSERT_OK(StartElection(ts_map_[leader_uuid], tablet_id, kTimeout));
-  ASSERT_OK(WaitUntilLeader(ts_map_[leader_uuid], tablet_id, kTimeout));
-
   auto active_ts_map = ts_map_;
   ASSERT_EQ(1, active_ts_map.erase(missing_replica_uuid));
 
@@ -132,9 +126,13 @@ TEST_F(RaftConsensusFailureDetectorIMCTest, TestFailureDetectorActivation)
{
   NO_FATALS(validate_failure_detector_status(active_ts_map));
 
   // Add a new non-voter.
-  ASSERT_OK(AddServer(ts_map_[leader_uuid], tablet_id,
-                      ts_map_[missing_replica_uuid],
-                      RaftPeerPB::NON_VOTER, kTimeout));
+  ASSERT_EVENTUALLY([&] {
+    itest::TServerDetails* leader;
+    ASSERT_OK(FindTabletLeader(active_ts_map, tablet_id, kTimeout, &leader));
+    ASSERT_OK(AddServer(leader, tablet_id,
+                        ts_map_[missing_replica_uuid],
+                        RaftPeerPB::NON_VOTER, kTimeout));
+  });
   ASSERT_OK(WaitForServersToAgree(kTimeout, ts_map_, tablet_id,
                                   /*minimum_index=*/ 2));
 


Mime
View raw message