kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From abu...@apache.org
Subject kudu git commit: KUDU-1592 Make warnings about FBM more ominous
Date Fri, 05 Oct 2018 18:41:09 GMT
Repository: kudu
Updated Branches:
  refs/heads/master b848488d9 -> f1f192ca3


KUDU-1592 Make warnings about FBM more ominous

The documentation warned about scalability issues with file block
manager, but it was rather lightly worded and so was the warning message
of a hole punch test failure.

This commit removes the actual flag necessary to enable FBM from the
hole punching tests' warning to force users look it up in the
documentation instead of blindly applying it, adds a note about it being
unsuitable for production use to the flag's description and expands the
warning in the troubleshooting section.

Change-Id: I15adf42dd7e6376a85cdb178ad460f239a4f87a1
Reviewed-on: http://gerrit.cloudera.org:8080/11579
Reviewed-by: Adar Dembo <adar@cloudera.com>
Reviewed-by: Andrew Wong <awong@cloudera.com>
Tested-by: Attila Bukor <abukor@apache.org>


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/f1f192ca
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/f1f192ca
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/f1f192ca

Branch: refs/heads/master
Commit: f1f192ca3abead4055e7ab6e6bba84d3d6e9866b
Parents: b848488
Author: Attila Bukor <abukor@apache.org>
Authored: Fri Oct 5 19:53:38 2018 +0200
Committer: Attila Bukor <abukor@apache.org>
Committed: Fri Oct 5 18:40:27 2018 +0000

----------------------------------------------------------------------
 docs/troubleshooting.adoc | 26 ++++++++++++++++++++------
 src/kudu/fs/data_dirs.cc  |  8 ++++++--
 src/kudu/fs/fs_manager.cc |  3 ++-
 3 files changed, 28 insertions(+), 9 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kudu/blob/f1f192ca/docs/troubleshooting.adoc
----------------------------------------------------------------------
diff --git a/docs/troubleshooting.adoc b/docs/troubleshooting.adoc
index 1bebe07..c3f8fbf 100644
--- a/docs/troubleshooting.adoc
+++ b/docs/troubleshooting.adoc
@@ -48,8 +48,12 @@ Error during hole punch test. The log block manager requires a
 filesystem with hole punching support such as ext4 or xfs. On el6,
 kernel version 2.6.32-358 or newer is required. To run without hole
 punching (at the cost of some efficiency and scalability), reconfigure
-Kudu with --block_manager=file. Refer to the Kudu documentation for more
-details. Raw error message follows.
+Kudu to use the file block manager. Refer to the Kudu documentation for
+more details. WARNING: the file block manager is not suitable for
+production use and should be used only for small-scale evaluation and
+development on systems where hole-punching is not available. It's
+impossible to switch between block managers after data is written to the
+server. Raw error message follows
 ----
 
 [NOTE]
@@ -70,8 +74,18 @@ adding the `--block_manager=file` flag to the commands you use to start
the mast
 and tablet servers. The file block manager does not scale as well as the log block
 manager.
 
-WARNING: The file block manager is known to scale and perform poorly, and should
-only be used for small-scale evaluation and development.
+[WARNING]
+====
+The file block manager is known to scale and perform poorly, and should
+only be used for small-scale evaluation and development, and only on systems
+where hole punching is unavailable.
+
+The file block manager uses one file per block. As multiple blocks are written
+for each rowset, the number of blocks can be very high, especially for actively
+written tablets. This can cause performance issues compared to the log block
+manager even with a small amount of data and it's *impossible to switch between
+block managers* without wiping and reinitializing the tablet servers.
+====
 
 [[disk_issues]]
 === Already present: FS layout already exists
@@ -224,7 +238,7 @@ jitter, while `lpeers` lists dispersion.
 
 
 [NOTE]
-****
+====
 .Using `chrony` for time synchronization
 
 Some operating systems offer `chrony` as an alternative to `ntpd` for network time
@@ -233,7 +247,7 @@ synchronization. Kudu has been tested most thoroughly using `ntpd` and
use of
 
 In order to use `chrony` for synchronization, `chrony.conf` must be configured
 with the `rtcsync` option.
-****
+====
 
 ==== NTP Configuration Best Practices
 

http://git-wip-us.apache.org/repos/asf/kudu/blob/f1f192ca/src/kudu/fs/data_dirs.cc
----------------------------------------------------------------------
diff --git a/src/kudu/fs/data_dirs.cc b/src/kudu/fs/data_dirs.cc
index 49139ea..af0cbec 100644
--- a/src/kudu/fs/data_dirs.cc
+++ b/src/kudu/fs/data_dirs.cc
@@ -136,8 +136,12 @@ const char kHolePunchErrorMsg[] =
     "filesystem with hole punching support such as ext4 or xfs. On el6, "
     "kernel version 2.6.32-358 or newer is required. To run without hole "
     "punching (at the cost of some efficiency and scalability), reconfigure "
-    "Kudu with --block_manager=file. Refer to the Kudu documentation for more "
-    "details. Raw error message follows";
+    "Kudu to use the file block manager. Refer to the Kudu documentation for "
+    "more details. WARNING: the file block manager is not suitable for "
+    "production use and should be used only for small-scale evaluation and "
+    "development on systems where hole-punching is not available. It's "
+    "impossible to switch between block managers after data is written to the "
+    "server. Raw error message follows";
 
 Status CheckHolePunch(Env* env, const string& path) {
   // Arbitrary constants.

http://git-wip-us.apache.org/repos/asf/kudu/blob/f1f192ca/src/kudu/fs/fs_manager.cc
----------------------------------------------------------------------
diff --git a/src/kudu/fs/fs_manager.cc b/src/kudu/fs/fs_manager.cc
index a9b954d..1255888 100644
--- a/src/kudu/fs/fs_manager.cc
+++ b/src/kudu/fs/fs_manager.cc
@@ -67,7 +67,8 @@ TAG_FLAG(enable_data_block_fsync, unsafe);
 
 #if defined(__linux__)
 DEFINE_string(block_manager, "log", "Which block manager to use for storage. "
-              "Valid options are 'file' and 'log'.");
+              "Valid options are 'file' and 'log'. The file block manager is not suitable
for "
+              "production use due to scaling limitations.");
 #else
 DEFINE_string(block_manager, "file", "Which block manager to use for storage. "
               "Only the file block manager is supported for non-Linux systems.");


Mime
View raw message