kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From danburk...@apache.org
Subject [1/2] kudu git commit: KUDU-1489: allow configuration of metadata dir
Date Thu, 18 Jan 2018 20:46:32 GMT
Repository: kudu
Updated Branches:
  refs/heads/master 1f346a1be -> 8bfc2a1e6


KUDU-1489: allow configuration of metadata dir

Metadata files currently reside in the first configured data directory,
which isn't always the best choice; this first drive, if not performant,
can act as a performance bottleneck. Often times the drive containing
the WALs is a better choice, as it is recommended to be the fastest.

This patch allows users to configure their metadata directory through
the new fs_metadata_dir flag. If empty, Kudu will check if a metadata
directory exists in the first member of fs_data_dirs, and if none
exists, Kudu will use fs_wal_dir as the root directory for metadata.
Otherwise, the flag will be honored verbatim.

Aside from the update to the directory location, codepaths that
previously assumed that the first data directory _must_ be healthy have
been updated. The remaining invariant is that at least a single data
directory must be healthy.

The following test changes are included:
- fs_manager-test: unit tests for various scenarios
- data_dirs-test: a test case is updated to show that we can now open
  the directory manager with a failed first data directory (previously
  this codepath would hit D/CHECK failures)
- mini_tablet_server-test: an end-to-end test is added to manually
  go back and forth between the old and new default metadata directories
- kudu-tool-test: a small test that tools still work, but only when the
  appropriate FS flags are provided

Change-Id: I375c6b2eb283db5fa9c956135d98252c8781f5db
Reviewed-on: http://gerrit.cloudera.org:8080/9027
Tested-by: Kudu Jenkins
Reviewed-by: Adar Dembo <adar@cloudera.com>
Reviewed-by: Todd Lipcon <todd@apache.org>


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/64926335
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/64926335
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/64926335

Branch: refs/heads/master
Commit: 64926335fe263b43f1493c03a91ea999759dfc71
Parents: 1f346a1
Author: Andrew Wong <awong@cloudera.com>
Authored: Wed Jan 10 14:36:52 2018 -0800
Committer: Todd Lipcon <todd@apache.org>
Committed: Thu Jan 18 20:08:08 2018 +0000

----------------------------------------------------------------------
 src/kudu/fs/block_manager_util.cc               |  22 ++--
 src/kudu/fs/data_dirs-test.cc                   |  15 +--
 src/kudu/fs/data_dirs.cc                        |  34 +++---
 src/kudu/fs/fs_manager-test.cc                  | 112 +++++++++++++++++--
 src/kudu/fs/fs_manager.cc                       |  56 +++++++---
 src/kudu/fs/fs_manager.h                        |  18 ++-
 .../external_mini_cluster_fs_inspector.cc       |  28 ++---
 src/kudu/tools/kudu-tool-test.cc                |  33 ++++++
 src/kudu/tools/tool_action_fs.cc                |  24 ++--
 src/kudu/tools/tool_action_local_replica.cc     |  33 ++++--
 src/kudu/tserver/mini_tablet_server-test.cc     |  59 ++++++++++
 11 files changed, 347 insertions(+), 87 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kudu/blob/64926335/src/kudu/fs/block_manager_util.cc
----------------------------------------------------------------------
diff --git a/src/kudu/fs/block_manager_util.cc b/src/kudu/fs/block_manager_util.cc
index 1920d2a..2a86ec8 100644
--- a/src/kudu/fs/block_manager_util.cc
+++ b/src/kudu/fs/block_manager_util.cc
@@ -166,16 +166,24 @@ Status PathInstanceMetadataFile::CheckIntegrity(
   // files, the (user-facing) error messages are reported in terms of data
   // directories, because UUIDs and instance files are internal details.
 
+  int first_healthy = -1;
+  for (int i = 0; i < instances.size(); i++) {
+    if (instances[i]->healthy()) {
+      first_healthy = i;
+      break;
+    }
+  }
+  if (first_healthy == -1) {
+    return Status::IOError("All data directories are unhealthy");
+  }
+
   // Map of instance UUID to path instance structure. Tracks duplicate UUIDs.
   unordered_map<string, PathInstanceMetadataFile*> uuids;
 
-  // Set of UUIDs specified in the path set of the first instance. All instances
-  // will be compared against this one to make sure all path sets match.
-  //
-  // Note that the first instance must be healthy, as it is the instance within
-  // the metadata root.
-  set<string> all_uuids(instances[0]->metadata()->path_set().all_uuids().begin(),
-                        instances[0]->metadata()->path_set().all_uuids().end());
+  // Set of UUIDs specified in the path set of the first healthy instance. All
+  // instances will be compared against it to make sure all path sets match.
+  set<string> all_uuids(instances[first_healthy]->metadata()->path_set().all_uuids().begin(),
+                        instances[first_healthy]->metadata()->path_set().all_uuids().end());
 
   if (all_uuids.size() != instances.size()) {
     return Status::IOError(

http://git-wip-us.apache.org/repos/asf/kudu/blob/64926335/src/kudu/fs/data_dirs-test.cc
----------------------------------------------------------------------
diff --git a/src/kudu/fs/data_dirs-test.cc b/src/kudu/fs/data_dirs-test.cc
index d35facc..e5ea7e2 100644
--- a/src/kudu/fs/data_dirs-test.cc
+++ b/src/kudu/fs/data_dirs-test.cc
@@ -397,29 +397,26 @@ TEST_F(DataDirManagerTest, TestOpenWithFailedDirs) {
   ASSERT_OK(DataDirManager::CreateNewForTests(
       env_, test_roots_, DataDirManagerOptions(), &dd_manager_));
 
-  // Kill the first non-metadata directory.
+  // Kill the first directory.
   FLAGS_crash_on_eio = false;
   FLAGS_env_inject_eio = 1.0;
-  FLAGS_env_inject_eio_globs = JoinPathSegments(test_roots_[1], "**");
+  FLAGS_env_inject_eio_globs = JoinPathSegments(test_roots_[0], "**");
 
   // The directory manager will successfully open with the single failed directory.
   ASSERT_OK(OpenDataDirManager());
   set<int> failed_dirs;
   ASSERT_EQ(1, dd_manager_->GetFailedDataDirs().size());
 
-  // Now fail almost all of the other directories, leaving the first intact.
-  for (int i = 2; i < kNumDirs; i++) {
-    // Completely change the failed directory and reopen the directory manager.
-    // The previously failed disks should durably failed.
+  // Now fail almost all of the other directories, leaving the last intact.
+  for (int i = 1; i < kNumDirs - 1; i++) {
     FLAGS_env_inject_eio_globs = Substitute("$0,$1", FLAGS_env_inject_eio_globs,
                                             JoinPathSegments(test_roots_[i], "**"));
   }
-  // The directory manager should still be aware of the previously failed
-  // disk(s), as well as the newly failed on.
+  // The directory manager should be aware of the new failures.
   ASSERT_OK(OpenDataDirManager());
   ASSERT_EQ(kNumDirs - 1, dd_manager_->GetFailedDataDirs().size());
 
-  // Ensure that when all data directories have failed, the server will crash.
+  // Ensure that when all data directories fail, the server will crash.
   FLAGS_env_inject_eio_globs = JoinStrings(JoinPathSegmentsV(test_roots_, "**"), ",");
   Status s = DataDirManager::OpenExistingForTests(env_, test_roots_,
       DataDirManagerOptions(), &dd_manager_);

http://git-wip-us.apache.org/repos/asf/kudu/blob/64926335/src/kudu/fs/data_dirs.cc
----------------------------------------------------------------------
diff --git a/src/kudu/fs/data_dirs.cc b/src/kudu/fs/data_dirs.cc
index 5f1c461..32ee476 100644
--- a/src/kudu/fs/data_dirs.cc
+++ b/src/kudu/fs/data_dirs.cc
@@ -527,15 +527,13 @@ Status DataDirManager::LoadInstances(
     unique_ptr<PathInstanceMetadataFile> instance(
         new PathInstanceMetadataFile(env_, opts_.block_manager_type, instance_filename));
     if (PREDICT_FALSE(!root.status.ok())) {
-      DCHECK_GT(i, 0);  // Guaranteed by FsManager::Init().
       instance->SetInstanceFailed(root.status);
     } else {
       // This may return OK and mark 'instance' as failed.
       Status s = instance->LoadFromDisk();
-      if (s.IsNotFound() && opts_.update_on_disk && i > 0) {
+      if (s.IsNotFound() && opts_.update_on_disk) {
         // A missing instance is only tolerated if we've been asked to add new
-        // data directories and this isn't the first one (i.e. the data
-        // directory used for metadata).
+        // data directories.
         missing_roots_tmp.emplace_back(root.path);
         continue;
       }
@@ -562,12 +560,17 @@ Status DataDirManager::LoadInstances(
     loaded_instances_tmp.emplace_back(std::move(instance));
   }
 
-  // Check that the first data directory (used for metadata) exists and is healthy.
+  // Check that at least a single data directory exists and is healthy.
   CHECK(!loaded_instances_tmp.empty());  // enforced above
-  if (!loaded_instances_tmp[0]->healthy()) {
-    return Status::IOError(Substitute(
-        "could not open directory manager; metadata directory failed: $0",
-        loaded_instances_tmp[0]->health_status().ToString()));
+  int num_healthy_instances = 0;
+  for (const auto& instance : loaded_instances_tmp) {
+    if (instance->healthy()) {
+      num_healthy_instances++;
+    }
+  }
+  if (num_healthy_instances == 0) {
+    return Status::IOError("could not open directory manager; all data "
+                           "directories failed");
   }
   missing_roots->swap(missing_roots_tmp);
   loaded_instances->swap(loaded_instances_tmp);
@@ -695,13 +698,18 @@ Status DataDirManager::Open() {
   };
 
   vector<DataDir*> unassigned_dirs;
+  int first_healthy = -1;
   // Assign a uuid index to each healthy instance.
-  for (const auto& dd : dds) {
+  for (int dir = 0; dir < dds.size(); dir++) {
+    const auto& dd = dds[dir];
     if (PREDICT_FALSE(!dd->instance()->healthy())) {
       // Keep track of failed directories so we can assign them UUIDs later.
       unassigned_dirs.push_back(dd.get());
       continue;
     }
+    if (first_healthy == -1) {
+      first_healthy = dir;
+    }
     const PathSetPB& path_set = dd->instance()->metadata()->path_set();
     int idx = -1;
     for (int i = 0; i < path_set.all_uuids_size(); i++) {
@@ -717,11 +725,11 @@ Status DataDirManager::Open() {
     }
     insert_to_maps(idx, path_set.uuid(), dd.get());
   }
+  CHECK_NE(first_healthy, -1); // Guaranteed by LoadInstances().
 
   // If the uuid index was not assigned, assign it to a failed directory. Use
-  // the 'all_uuids' of the first data directory, as it is guaranteed to be
-  // healthy.
-  PathSetPB path_set = dds[0]->instance()->metadata()->path_set();
+  // the path set from the first healthy instance.
+  PathSetPB path_set = dds[first_healthy]->instance()->metadata()->path_set();
   int failed_dir_idx = 0;
   for (int uuid_idx = 0; uuid_idx < path_set.all_uuids_size(); uuid_idx++) {
     if (!ContainsKey(uuid_by_idx, uuid_idx)) {

http://git-wip-us.apache.org/repos/asf/kudu/blob/64926335/src/kudu/fs/fs_manager-test.cc
----------------------------------------------------------------------
diff --git a/src/kudu/fs/fs_manager-test.cc b/src/kudu/fs/fs_manager-test.cc
index 5df0f57..8157aae 100644
--- a/src/kudu/fs/fs_manager-test.cc
+++ b/src/kudu/fs/fs_manager-test.cc
@@ -237,6 +237,109 @@ TEST_F(FsManagerTestBase, TestFormatWithSpecificUUID) {
   ASSERT_EQ(uuid, fs_manager()->uuid());
 }
 
+TEST_F(FsManagerTestBase, TestMetadataDirInWALRoot) {
+  // By default, the FsManager should put metadata in the wal root.
+  FsManagerOpts opts;
+  opts.wal_root = GetTestPath("wal");
+  opts.data_roots = { GetTestPath("data") };
+  ReinitFsManagerWithOpts(opts);
+  ASSERT_OK(fs_manager()->CreateInitialFileSystemLayout());
+  ASSERT_OK(fs_manager()->Open());
+  ASSERT_STR_CONTAINS(fs_manager()->GetTabletMetadataDir(),
+                      JoinPathSegments("wal", FsManager::kTabletMetadataDirName));
+
+  // Reinitializing the FS layout with any other configured metadata root
+  // should fail, as a non-empty metadata root will be used verbatim.
+  opts.metadata_root = GetTestPath("asdf");
+  ReinitFsManagerWithOpts(opts);
+  Status s = fs_manager()->Open();
+  ASSERT_TRUE(s.IsNotFound()) << s.ToString();
+
+  // The above comment also applies to the default value before Kudu 1.6: the
+  // first configured data directory. Let's check that too.
+  opts.metadata_root = opts.data_roots[0];
+  ReinitFsManagerWithOpts(opts);
+  s = fs_manager()->Open();
+  ASSERT_TRUE(s.IsNotFound()) << s.ToString();
+
+  // We should be able to verify that the metadata is in the WAL root.
+  opts.metadata_root = opts.wal_root;
+  ReinitFsManagerWithOpts(opts);
+  ASSERT_OK(fs_manager()->Open());
+}
+
+TEST_F(FsManagerTestBase, TestMetadataDirInDataRoot) {
+  FsManagerOpts opts;
+  opts.wal_root = GetTestPath("wal");
+  opts.data_roots = { GetTestPath("data1") };
+
+  // Creating a brand new FS layout configured with metadata in the first data
+  // directory emulates the default behavior in Kudu 1.6 and below.
+  opts.metadata_root = opts.data_roots[0];
+  ReinitFsManagerWithOpts(opts);
+  ASSERT_OK(fs_manager()->CreateInitialFileSystemLayout());
+  ASSERT_OK(fs_manager()->Open());
+  const string& meta_root_suffix = JoinPathSegments("data1", FsManager::kTabletMetadataDirName);
+  ASSERT_STR_CONTAINS(fs_manager()->GetTabletMetadataDir(), meta_root_suffix);
+
+  // Opening the FsManager with an empty fs_metadata_dir flag should account
+  // for the old default and use the first data directory for metadata.
+  opts.metadata_root.clear();
+  ReinitFsManagerWithOpts(opts);
+  ASSERT_OK(fs_manager()->Open());
+  ASSERT_STR_CONTAINS(fs_manager()->GetTabletMetadataDir(), meta_root_suffix);
+
+  // Now let's test adding data directories with metadata in the data root.
+  // Adding data directories is not supported by the file block manager.
+  if (FLAGS_block_manager == "file") {
+    LOG(INFO) << "Skipping the rest of test, file block manager not supported";
+    return;
+  }
+
+  // Adding a data dir to the front of the FS root list (i.e. such that the
+  // metadata root is no longer at the front) will prevent Kudu from starting.
+  opts.data_roots = { GetTestPath("data2"), GetTestPath("data1") };
+  opts.update_on_disk = true;
+  ReinitFsManagerWithOpts(opts);
+  Status s = fs_manager()->Open();
+  ASSERT_STR_CONTAINS(s.ToString(), "could not verify required directory");
+  ASSERT_TRUE(s.IsNotFound());
+  ASSERT_FALSE(env_->FileExists(opts.data_roots[0]));
+  ASSERT_TRUE(env_->FileExists(opts.data_roots[1]));
+
+  // Now allow the reordering by specifying the expected metadata root.
+  opts.metadata_root = opts.data_roots[1];
+  ReinitFsManagerWithOpts(opts);
+  ASSERT_OK(fs_manager()->Open());
+  ASSERT_STR_CONTAINS(fs_manager()->GetTabletMetadataDir(), meta_root_suffix);
+}
+
+TEST_F(FsManagerTestBase, TestIsolatedMetadataDir) {
+  FsManagerOpts opts;
+  opts.wal_root = GetTestPath("wal");
+  opts.data_roots = { GetTestPath("data") };
+
+  // Creating a brand new FS layout configured to a directory outside the WAL
+  // or data directories is supported.
+  opts.metadata_root = GetTestPath("asdf");
+  ReinitFsManagerWithOpts(opts);
+  ASSERT_OK(fs_manager()->CreateInitialFileSystemLayout());
+  ASSERT_OK(fs_manager()->Open());
+  ASSERT_STR_CONTAINS(fs_manager()->GetTabletMetadataDir(),
+                      JoinPathSegments("asdf", FsManager::kTabletMetadataDirName));
+  ASSERT_NE(DirName(fs_manager()->GetTabletMetadataDir()),
+            DirName(fs_manager()->GetWalsRootDir()));
+  ASSERT_NE(DirName(fs_manager()->GetTabletMetadataDir()),
+            DirName(fs_manager()->GetDataRootDirs()[0]));
+
+  // If the user henceforth forgets to specify the metadata root, the FsManager
+  // will fail to open.
+  opts.metadata_root.clear();
+  ReinitFsManagerWithOpts(opts);
+  Status s = fs_manager()->Open();
+  ASSERT_TRUE(s.IsNotFound()) << s.ToString();
+}
+
 Status CountTmpFiles(Env* env, const string& path, const vector<string>& children,
                      unordered_set<string>* checked_dirs, size_t* count) {
   vector<string> sub_objects;
@@ -459,17 +562,10 @@ TEST_F(FsManagerTestBase, TestAddDataDirs) {
   ASSERT_TRUE(s.IsIOError());
   ASSERT_STR_CONTAINS(s.ToString(), "could not verify integrity of files");
 
-  // Try to open with a new data dir at the front of the list; this should fail.
+  // We should be able to add new directories anywhere in the list.
   const string new_path2 = GetTestPath("new_path2");
   opts.data_roots = { new_path2, fs_root_, new_path1 };
   ReinitFsManagerWithOpts(opts);
-  s = fs_manager()->Open();
-  ASSERT_TRUE(s.IsNotFound());
-  ASSERT_STR_CONTAINS(s.ToString(), "could not verify required directory");
-
-  // But adding a new data dir elsewhere in the list is OK.
-  opts.data_roots = { fs_root_, new_path2, new_path1 };
-  ReinitFsManagerWithOpts(opts);
   ASSERT_OK(fs_manager()->Open());
   ASSERT_EQ(3, fs_manager()->dd_manager()->GetDataDirs().size());
 

http://git-wip-us.apache.org/repos/asf/kudu/blob/64926335/src/kudu/fs/fs_manager.cc
----------------------------------------------------------------------
diff --git a/src/kudu/fs/fs_manager.cc b/src/kudu/fs/fs_manager.cc
index 9bbacb0..df61bf6 100644
--- a/src/kudu/fs/fs_manager.cc
+++ b/src/kudu/fs/fs_manager.cc
@@ -20,7 +20,6 @@
 #include <cinttypes>
 #include <ctime>
 #include <iostream>
-#include <set>
 #include <unordered_map>
 #include <unordered_set>
 #include <utility>
@@ -90,6 +89,13 @@ DEFINE_string(fs_data_dirs, "",
               "is not specified, fs_wal_dir will be used as the sole data "
               "block directory.");
 TAG_FLAG(fs_data_dirs, stable);
+DEFINE_string(fs_metadata_dir, "",
+              "Directory with metadata. If this is not specified, for "
+              "compatibility with Kudu 1.6 and below, Kudu will check the "
+              "first entry of fs_data_dirs for metadata and use it as the "
+              "metadata directory if any exists. If none exists, fs_wal_dir "
+              "will be used as the metadata directory.");
+TAG_FLAG(fs_metadata_dir, stable);
 
 using kudu::fs::BlockManagerOptions;
 using kudu::fs::CreateBlockOptions;
@@ -128,6 +134,7 @@ const char *FsManager::kConsensusMetadataDirName = "consensus-meta";
 
 FsManagerOpts::FsManagerOpts()
   : wal_root(FLAGS_fs_wal_dir),
+    metadata_root(FLAGS_fs_metadata_dir),
     block_manager_type(FLAGS_block_manager),
     read_only(false),
     update_on_disk(false) {
@@ -180,6 +187,12 @@ Status FsManager::Init() {
   unordered_set<string> all_roots = { opts_.wal_root };
   all_roots.insert(opts_.data_roots.begin(), opts_.data_roots.end());
 
+  // If the metadata root not set, Kudu will either use the wal root or the
+  // first data root, in which case we needn't canonicalize additional roots.
+  if (!opts_.metadata_root.empty()) {
+    all_roots.insert(opts_.metadata_root);
+  }
+
   // Build a map of original root --> canonicalized root, sanitizing each
   // root as we go and storing the canonicalization status.
   typedef unordered_map<string, CanonicalizedRootAndStatus> RootMap;
@@ -219,8 +232,8 @@ Status FsManager::Init() {
   // All done, use the map to set the canonicalized state.
 
   canonicalized_wal_fs_root_ = FindOrDie(canonicalized_roots, opts_.wal_root);
+  unordered_set<string> unique_roots;
   if (!opts_.data_roots.empty()) {
-    unordered_set<string> unique_roots;
     for (const string& data_fs_root : opts_.data_roots) {
       const auto& root = FindOrDie(canonicalized_roots, data_fs_root);
       if (InsertIfNotPresent(&unique_roots, root.path)) {
@@ -228,18 +241,38 @@ Status FsManager::Init() {
         canonicalized_all_fs_roots_.emplace_back(root);
       }
     }
-    canonicalized_metadata_fs_root_ = FindOrDie(canonicalized_roots, opts_.data_roots[0]);
-    if (!ContainsKey(unique_roots, canonicalized_wal_fs_root_.path)) {
-      canonicalized_all_fs_roots_.emplace_back(canonicalized_wal_fs_root_);
-    }
   } else {
     LOG(INFO) << "Data directories (fs_data_dirs) not provided";
     LOG(INFO) << "Using write-ahead log directory (fs_wal_dir) as data directory";
-    canonicalized_metadata_fs_root_ = canonicalized_wal_fs_root_;
     canonicalized_data_fs_roots_.emplace_back(canonicalized_wal_fs_root_);
+  }
+  if (InsertIfNotPresent(&unique_roots, canonicalized_wal_fs_root_.path)) {
     canonicalized_all_fs_roots_.emplace_back(canonicalized_wal_fs_root_);
   }
 
+  // Decide on a metadata root to use.
+  if (opts_.metadata_root.empty()) {
+    // Check the first data root for metadata.
+    const string meta_dir_in_data_root = JoinPathSegments(canonicalized_data_fs_roots_[0].path,
+                                                          kTabletMetadataDirName);
+    // If there is already metadata in the first data root, use it. Otherwise,
+    // use the WAL root.
+    LOG(INFO) << "Metadata directory not provided";
+    if (env_->FileExists(meta_dir_in_data_root)) {
+      canonicalized_metadata_fs_root_ = canonicalized_data_fs_roots_[0];
+      LOG(INFO) << "Using existing metadata directory in first data directory";
+    } else {
+      canonicalized_metadata_fs_root_ = canonicalized_wal_fs_root_;
+      LOG(INFO) << "Using write-ahead log directory (fs_wal_dir) as metadata directory";
+    }
+  } else {
+    // Keep track of the explicitly-defined metadata root.
+    canonicalized_metadata_fs_root_ = FindOrDie(canonicalized_roots, opts_.metadata_root);
+    if (InsertIfNotPresent(&unique_roots, canonicalized_metadata_fs_root_.path)) {
+      canonicalized_all_fs_roots_.emplace_back(canonicalized_metadata_fs_root_);
+    }
+  }
+
   // The server cannot start if the WAL root or metadata root failed to
   // canonicalize.
   const string& wal_root = canonicalized_wal_fs_root_.path;
@@ -385,15 +418,6 @@ Status FsManager::Open(FsReport* report) {
     RETURN_NOT_OK(block_manager_->Open(report));
   }
 
-  // Before returning, ensure the metadata directory has not failed.
-  // TODO(awong): avoid this single point of failure by spreading metadata
-  // across directories.
-  int metadata_idx;
-  CHECK(dd_manager_->FindUuidIndexByRoot(canonicalized_metadata_fs_root_.path, &metadata_idx));
-  if (ContainsKey(dd_manager_->GetFailedDataDirs(), metadata_idx)) {
-    return Status::IOError("Cannot open the FS layout; metadata directory failed");
-  }
-
   if (FLAGS_enable_data_block_fsync) {
     // Files/directories created by the directory manager in the fs roots have
     // been synchronized, so now is a good time to sync the roots themselves.

http://git-wip-us.apache.org/repos/asf/kudu/blob/64926335/src/kudu/fs/fs_manager.h
----------------------------------------------------------------------
diff --git a/src/kudu/fs/fs_manager.h b/src/kudu/fs/fs_manager.h
index 2c0fc21..dc158bd 100644
--- a/src/kudu/fs/fs_manager.h
+++ b/src/kudu/fs/fs_manager.h
@@ -59,6 +59,10 @@ namespace itest {
 class ExternalMiniClusterFsInspector;
 } // namespace itest
 
+namespace tserver {
+class MiniTabletServerTest_TestFsLayoutEndToEnd_Test;
+} // namespace tserver
+
 struct FsManagerOpts {
   // Creates a new FsManagerOpts with default values.
   FsManagerOpts();
@@ -82,9 +86,16 @@ struct FsManagerOpts {
   // The directory root where WALs will be stored. Cannot be empty.
   std::string wal_root;
 
-  // The directory root where data blocks will be stored. Cannot be empty.
+  // The directory root where data blocks will be stored. If empty, Kudu will
+  // use the WAL root.
   std::vector<std::string> data_roots;
 
+  // The directory root where metadata will be stored. If empty, Kudu will use
+  // the WAL root, or the first configured data root if metadata already exists
+  // in it from a previous deployment (the only option in Kudu 1.6 and below
+  // was to use the first data root).
+  std::string metadata_root;
+
   // The block manager type. Must be either "file" or "log".
   // Defaults to the value of FLAGS_block_manager.
   std::string block_manager_type;
@@ -248,9 +259,14 @@ class FsManager {
 
  private:
   FRIEND_TEST(FsManagerTestBase, TestDuplicatePaths);
+  FRIEND_TEST(FsManagerTestBase, TestMetadataDirInWALRoot);
+  FRIEND_TEST(FsManagerTestBase, TestMetadataDirInDataRoot);
+  FRIEND_TEST(FsManagerTestBase, TestIsolatedMetadataDir);
+  FRIEND_TEST(tserver::MiniTabletServerTest, TestFsLayoutEndToEnd);
   friend class itest::ExternalMiniClusterFsInspector; // for access to directory names
 
   // Initializes, sanitizes, and canonicalizes the filesystem roots.
+  // Determines the correct filesystem root for tablet-specific metadata.
   Status Init();
 
   // Select and create an instance of the appropriate block manager.

http://git-wip-us.apache.org/repos/asf/kudu/blob/64926335/src/kudu/integration-tests/external_mini_cluster_fs_inspector.cc
----------------------------------------------------------------------
diff --git a/src/kudu/integration-tests/external_mini_cluster_fs_inspector.cc b/src/kudu/integration-tests/external_mini_cluster_fs_inspector.cc
index f22f12f..bc69a80 100644
--- a/src/kudu/integration-tests/external_mini_cluster_fs_inspector.cc
+++ b/src/kudu/integration-tests/external_mini_cluster_fs_inspector.cc
@@ -100,8 +100,8 @@ vector<string> ExternalMiniClusterFsInspector::ListTablets() {
 }
 
 vector<string> ExternalMiniClusterFsInspector::ListTabletsOnTS(int index) {
-  string data_dir = cluster_->tablet_server(index)->data_dirs()[0];
-  string meta_dir = JoinPathSegments(data_dir, FsManager::kTabletMetadataDirName);
+  string wal_dir = cluster_->tablet_server(index)->wal_dir();
+  string meta_dir = JoinPathSegments(wal_dir, FsManager::kTabletMetadataDirName);
   vector<string> tablets;
   CHECK_OK(ListFilesInDir(meta_dir, &tablets));
   return tablets;
@@ -142,22 +142,22 @@ int ExternalMiniClusterFsInspector::CountReplicasInMetadataDirs() {
   // tablet servers isn't easy.
   int count = 0;
   for (int i = 0; i < cluster_->num_tablet_servers(); i++) {
-    string data_dir = cluster_->tablet_server(i)->data_dirs()[0];
-    count += CountFilesInDir(JoinPathSegments(data_dir, FsManager::kTabletMetadataDirName));
+    string wal_dir = cluster_->tablet_server(i)->wal_dir();
+    count += CountFilesInDir(JoinPathSegments(wal_dir, FsManager::kTabletMetadataDirName));
   }
   return count;
 }
 
 Status ExternalMiniClusterFsInspector::CheckNoDataOnTS(int index) {
-  string data_dir = cluster_->tablet_server(index)->data_dir();
-  if (CountFilesInDir(JoinPathSegments(data_dir, FsManager::kTabletMetadataDirName)) >
0) {
-    return Status::IllegalState("tablet metadata blocks still exist", data_dir);
+  const string& wal_dir = cluster_->tablet_server(index)->wal_dir();
+  if (CountFilesInDir(JoinPathSegments(wal_dir, FsManager::kTabletMetadataDirName)) >
0) {
+    return Status::IllegalState("tablet metadata blocks still exist", wal_dir);
   }
   if (CountWALFilesOnTS(index) > 0) {
-    return Status::IllegalState("wals still exist", data_dir);
+    return Status::IllegalState("wals still exist", wal_dir);
   }
-  if (CountFilesInDir(JoinPathSegments(data_dir, FsManager::kConsensusMetadataDirName)) >
0) {
-    return Status::IllegalState("consensus metadata still exists", data_dir);
+  if (CountFilesInDir(JoinPathSegments(wal_dir, FsManager::kConsensusMetadataDirName)) >
0) {
+    return Status::IllegalState("consensus metadata still exists", wal_dir);
   }
   return Status::OK();;
 }
@@ -171,8 +171,8 @@ Status ExternalMiniClusterFsInspector::CheckNoData() {
 
 string ExternalMiniClusterFsInspector::GetTabletSuperBlockPathOnTS(int ts_index,
                                                                    const string& tablet_id)
const {
-  string data_dir = cluster_->tablet_server(ts_index)->data_dirs()[0];
-  string meta_dir = JoinPathSegments(data_dir, FsManager::kTabletMetadataDirName);
+  string wal_dir = cluster_->tablet_server(ts_index)->wal_dir();
+  string meta_dir = JoinPathSegments(wal_dir, FsManager::kTabletMetadataDirName);
   return JoinPathSegments(meta_dir, tablet_id);
 }
 
@@ -192,8 +192,8 @@ int64_t ExternalMiniClusterFsInspector::GetTabletSuperBlockMTimeOrDie(int
ts_ind
 
 string ExternalMiniClusterFsInspector::GetConsensusMetadataPathOnTS(int index,
                                                                     const string& tablet_id)
const {
-  string data_dir = cluster_->tablet_server(index)->data_dirs()[0];
-  string cmeta_dir = JoinPathSegments(data_dir, FsManager::kConsensusMetadataDirName);
+  string wal_dir = cluster_->tablet_server(index)->wal_dir();
+  string cmeta_dir = JoinPathSegments(wal_dir, FsManager::kConsensusMetadataDirName);
   return JoinPathSegments(cmeta_dir, tablet_id);
 }
 

http://git-wip-us.apache.org/repos/asf/kudu/blob/64926335/src/kudu/tools/kudu-tool-test.cc
----------------------------------------------------------------------
diff --git a/src/kudu/tools/kudu-tool-test.cc b/src/kudu/tools/kudu-tool-test.cc
index 2798898..493c60d 100644
--- a/src/kudu/tools/kudu-tool-test.cc
+++ b/src/kudu/tools/kudu-tool-test.cc
@@ -2209,5 +2209,38 @@ TEST_F(ToolTest, TestFsAddDataDirEndToEnd) {
   ASSERT_GT(disk_space_used_after, disk_space_used_before);
 }
 
+TEST_F(ToolTest, TestDumpFSWithNonDefaultMetadataDir) {
+  const string kTestDir = GetTestPath("test");
+  ASSERT_OK(env_->CreateDir(kTestDir));
+  string uuid;
+  FsManagerOpts opts;
+  {
+    opts.wal_root = JoinPathSegments(kTestDir, "wal");
+    opts.metadata_root = JoinPathSegments(kTestDir, "meta");
+    FsManager fs(env_, opts);
+    ASSERT_OK(fs.CreateInitialFileSystemLayout());
+    ASSERT_OK(fs.Open());
+    uuid = fs.uuid();
+  }
+  // Because a non-default metadata directory was specified, users must provide
+  // enough arguments to open the FsManager, or else FS tools will not work.
+  // The tool will fail in its own process. Catch its output.
+  string stderr;
+  Status s = RunTool(Substitute("fs dump uuid --fs_wal_dir=$0", opts.wal_root),
+                    nullptr, &stderr, {}, {});
+  ASSERT_TRUE(s.IsRuntimeError());
+  ASSERT_STR_CONTAINS(s.ToString(), "process exited with non-zero status");
+  SCOPED_TRACE(stderr);
+  ASSERT_STR_CONTAINS(stderr, "could not verify required directory");
+
+  // Providing the necessary arguments, the tool should work.
+  string stdout;
+  NO_FATALS(RunActionStdoutString(Substitute(
+      "fs dump uuid --fs_wal_dir=$0 --fs_metadata_dir=$1",
+      opts.wal_root, opts.metadata_root), &stdout));
+  SCOPED_TRACE(stdout);
+  ASSERT_EQ(uuid, stdout);
+}
+
 } // namespace tools
 } // namespace kudu

http://git-wip-us.apache.org/repos/asf/kudu/blob/64926335/src/kudu/tools/tool_action_fs.cc
----------------------------------------------------------------------
diff --git a/src/kudu/tools/tool_action_fs.cc b/src/kudu/tools/tool_action_fs.cc
index 352a47d..16e1de5 100644
--- a/src/kudu/tools/tool_action_fs.cc
+++ b/src/kudu/tools/tool_action_fs.cc
@@ -759,8 +759,9 @@ static unique_ptr<Mode> BuildFsDumpMode() {
       .ExtraDescription("This interprets the contents of a CFile-formatted block "
                         "and outputs the decoded row data.")
       .AddRequiredParameter({ "block_id", "block identifier" })
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("print_meta")
       .AddOptionalParameter("print_rows")
       .Build();
@@ -771,22 +772,25 @@ static unique_ptr<Mode> BuildFsDumpMode() {
       .ExtraDescription("This performs no parsing or interpretation of the data stored "
                         "in the block but rather outputs its binary contents directly.")
       .AddRequiredParameter({ "block_id", "block identifier" })
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .Build();
 
   unique_ptr<Action> dump_tree =
       ActionBuilder("tree", &DumpFsTree)
       .Description("Dump the tree of a Kudu filesystem")
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .Build();
 
   unique_ptr<Action> dump_uuid =
       ActionBuilder("uuid", &DumpUuid)
       .Description("Dump the UUID of a Kudu filesystem")
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .Build();
 
   return ModeBuilder("dump")
@@ -802,16 +806,18 @@ unique_ptr<Mode> BuildFsMode() {
   unique_ptr<Action> check =
       ActionBuilder("check", &Check)
       .Description("Check a Kudu filesystem for inconsistencies")
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("repair")
       .Build();
 
   unique_ptr<Action> format =
       ActionBuilder("format", &Format)
       .Description("Format a new Kudu filesystem")
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("uuid")
       .Build();
 
@@ -819,8 +825,9 @@ unique_ptr<Mode> BuildFsMode() {
       ActionBuilder("update_dirs", &Update)
       .Description("Updates the set of data directories in an existing Kudu filesystem")
       .ExtraDescription("Cannot currently be used to remove data directories")
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .Build();
 
   unique_ptr<Action> list =
@@ -833,8 +840,9 @@ unique_ptr<Mode> BuildFsMode() {
                         "Note: adding any of the 'cfile' fields to --columns will cause "
                         "the tool to read on-disk metadata for each CFile in the result set,
"
                         "which could require large amounts of I/O when many results are returned.")
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("table_id")
       .AddOptionalParameter("tablet_id")
       .AddOptionalParameter("rowset_id")

http://git-wip-us.apache.org/repos/asf/kudu/blob/64926335/src/kudu/tools/tool_action_local_replica.cc
----------------------------------------------------------------------
diff --git a/src/kudu/tools/tool_action_local_replica.cc b/src/kudu/tools/tool_action_local_replica.cc
index 784def5..885474c 100644
--- a/src/kudu/tools/tool_action_local_replica.cc
+++ b/src/kudu/tools/tool_action_local_replica.cc
@@ -883,16 +883,18 @@ unique_ptr<Mode> BuildDumpMode() {
       ActionBuilder("block_ids", &DumpBlockIdsForLocalReplica)
       .Description("Dump the IDs of all blocks belonging to a local replica")
       .AddRequiredParameter({ kTabletIdArg, kTabletIdArgDesc })
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .Build();
 
   unique_ptr<Action> dump_meta =
       ActionBuilder("meta", &DumpMeta)
       .Description("Dump the metadata of a local replica")
       .AddRequiredParameter({ kTabletIdArg, kTabletIdArgDesc })
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .Build();
 
   unique_ptr<Action> dump_rowset =
@@ -900,8 +902,9 @@ unique_ptr<Mode> BuildDumpMode() {
       .Description("Dump the rowset contents of a local replica")
       .AddRequiredParameter({ kTabletIdArg, kTabletIdArgDesc })
       .AddOptionalParameter("dump_data")
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("metadata_only")
       .AddOptionalParameter("nrows")
       .AddOptionalParameter("rowset_index")
@@ -912,8 +915,9 @@ unique_ptr<Mode> BuildDumpMode() {
       .Description("Dump all WAL (write-ahead log) segments of "
         "a local replica")
       .AddRequiredParameter({ kTabletIdArg, kTabletIdArgDesc })
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("print_entries")
       .AddOptionalParameter("print_meta")
       .AddOptionalParameter("truncate_data")
@@ -936,8 +940,9 @@ unique_ptr<Mode> BuildLocalReplicaMode() {
       .Description("Print all tablet replica peer UUIDs found in a "
         "tablet's Raft configuration")
       .AddRequiredParameter({ kTabletIdArg, kTabletIdArgDesc })
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .Build();
 
   unique_ptr<Action> rewrite_raft_config =
@@ -947,8 +952,9 @@ unique_ptr<Mode> BuildLocalReplicaMode() {
       .AddRequiredVariadicParameter({
         "peers", "List of peers where each peer is of "
         "form 'uuid:hostname:port'" })
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .Build();
 
   unique_ptr<Action> set_term =
@@ -957,8 +963,9 @@ unique_ptr<Mode> BuildLocalReplicaMode() {
       .AddRequiredParameter({ kTabletIdArg, kTabletIdArgDesc })
       .AddRequiredParameter({ kTermArg, "the new raft term (must be greater "
         "than the current term)" })
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .Build();
 
   unique_ptr<Mode> cmeta =
@@ -976,15 +983,17 @@ unique_ptr<Mode> BuildLocalReplicaMode() {
       .AddRequiredParameter({ kTabletIdArg, kTabletIdArgDesc })
       .AddRequiredParameter({ "source", "Source RPC address of "
         "form hostname:port" })
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .Build();
 
   unique_ptr<Action> list =
       ActionBuilder("list", &ListLocalReplicas)
       .Description("Show list of tablet replicas in the local filesystem")
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("list_detail")
       .Build();
 
@@ -993,8 +1002,9 @@ unique_ptr<Mode> BuildLocalReplicaMode() {
       .Description("Delete a tablet replica from the local filesystem. "
           "By default, leaves a tombstone record.")
       .AddRequiredParameter({ kTabletIdArg, kTabletIdArgDesc })
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("clean_unsafe")
       .Build();
 
@@ -1002,8 +1012,9 @@ unique_ptr<Mode> BuildLocalReplicaMode() {
       ActionBuilder("data_size", &SummarizeDataSize)
       .Description("Summarize the data size/space usage of the given local replica(s).")
       .AddRequiredParameter({ kTabletIdGlobArg, kTabletIdGlobArgDesc })
-      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("fs_data_dirs")
+      .AddOptionalParameter("fs_metadata_dir")
+      .AddOptionalParameter("fs_wal_dir")
       .AddOptionalParameter("format")
       .Build();
 

http://git-wip-us.apache.org/repos/asf/kudu/blob/64926335/src/kudu/tserver/mini_tablet_server-test.cc
----------------------------------------------------------------------
diff --git a/src/kudu/tserver/mini_tablet_server-test.cc b/src/kudu/tserver/mini_tablet_server-test.cc
index 34237be..d50a8f4 100644
--- a/src/kudu/tserver/mini_tablet_server-test.cc
+++ b/src/kudu/tserver/mini_tablet_server-test.cc
@@ -16,6 +16,7 @@
 // under the License.
 
 #include <memory>
+#include <string>
 #include <vector>
 
 #include <gtest/gtest.h>
@@ -23,6 +24,7 @@
 #include "kudu/fs/fs_manager.h"
 #include "kudu/tserver/mini_tablet_server.h"
 #include "kudu/tserver/tablet_server.h"
+#include "kudu/util/env.h"
 #include "kudu/util/net/net_util.h"
 #include "kudu/util/path_util.h"
 #include "kudu/util/status.h"
@@ -32,6 +34,7 @@
 namespace kudu {
 namespace tserver {
 
+using std::string;
 using std::unique_ptr;
 
 class MiniTabletServerTest : public KuduTest {};
@@ -50,5 +53,61 @@ TEST_F(MiniTabletServerTest, TestMultiDirServer) {
   ASSERT_EQ(kNumDataDirs, fs_manager->GetDataRootDirs().size());
 }
 
+// Test opening the FS layout with metadata in the WAL root, moving it to the
+// first data root, and back.
+//
+// Users may want to move the metadata directory from the current default (the
+// WAL root) to the old default (the first data root), or may be using the old
+// default and may want to abide by the current default. Both cases are tested.
+TEST_F(MiniTabletServerTest, TestFsLayoutEndToEnd) {
+  // Use multiple data dirs, otherwise the mini tserver will colocated the WALs
+  // and data dir.
+  const int kNumDataDirs = 2;
+  unique_ptr<MiniTabletServer> mini_server;
+  FsManager* fs_manager;
+  const auto& reset_mini_tserver = [&] {
+    mini_server.reset(new MiniTabletServer(GetTestPath("TServer"),
+        HostPort("127.0.0.1", 0), kNumDataDirs));
+    ASSERT_OK(mini_server->Start());
+    fs_manager = mini_server->server()->fs_manager();
+  };
+  reset_mini_tserver();
+
+  // By default, the metadata directory will be placed in the WAL root.
+  const string& tmeta_dir_in_wal_root = fs_manager->GetTabletMetadataDir();
+  const string& cmeta_dir_in_wal_root = fs_manager->GetConsensusMetadataDir();
+  ASSERT_STR_CONTAINS(DirName(tmeta_dir_in_wal_root), "wal");
+  ASSERT_STR_CONTAINS(DirName(cmeta_dir_in_wal_root), "wal");
+
+  // Now move the metadata directories into the first directory, emulating the
+  // behavior in Kudu 1.6 and below.
+  const string data_root = DirName(fs_manager->GetDataRootDirs()[0]);
+  const string tmeta_dir_in_data_root = JoinPathSegments(
+      data_root, FsManager::kTabletMetadataDirName);
+  const string cmeta_dir_in_data_root = JoinPathSegments(
+      data_root, FsManager::kConsensusMetadataDirName);
+  mini_server->Shutdown();
+  ASSERT_OK(env_->RenameFile(tmeta_dir_in_wal_root, tmeta_dir_in_data_root));
+  ASSERT_OK(env_->RenameFile(cmeta_dir_in_wal_root, cmeta_dir_in_data_root));
+
+  // Upon restarting the server, since metadata directories already exist, Kudu
+  // will take them as is.
+  reset_mini_tserver();
+  ASSERT_EQ(tmeta_dir_in_data_root, fs_manager->GetTabletMetadataDir());
+  ASSERT_EQ(cmeta_dir_in_data_root, fs_manager->GetConsensusMetadataDir());
+  ASSERT_FALSE(env_->FileExists(tmeta_dir_in_wal_root));
+  ASSERT_FALSE(env_->FileExists(cmeta_dir_in_wal_root));
+
+  // Let's move back to the newer default and place metadata in the WAL root.
+  mini_server->Shutdown();
+  ASSERT_OK(env_->RenameFile(tmeta_dir_in_data_root, tmeta_dir_in_wal_root));
+  ASSERT_OK(env_->RenameFile(cmeta_dir_in_data_root, cmeta_dir_in_wal_root));
+  reset_mini_tserver();
+  ASSERT_EQ(tmeta_dir_in_wal_root, fs_manager->GetTabletMetadataDir());
+  ASSERT_EQ(cmeta_dir_in_wal_root, fs_manager->GetConsensusMetadataDir());
+  ASSERT_FALSE(env_->FileExists(tmeta_dir_in_data_root));
+  ASSERT_FALSE(env_->FileExists(cmeta_dir_in_data_root));
+}
+
 }  // namespace tserver
 }  // namespace kudu


Mime
View raw message