kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From abu...@apache.org
Subject [kudu] 03/04: KUDU-2195. Add additional gflag to force sync of consensus metadata
Date Wed, 09 Jan 2019 18:11:12 GMT
This is an automated email from the ASF dual-hosted git repository.

abukor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git

commit 5e334de24e016192413c23ea2554d49406df1985
Author: Mike Percy <mpercy@apache.org>
AuthorDate: Tue Jan 8 17:01:52 2019 -0800

    KUDU-2195. Add additional gflag to force sync of consensus metadata
    
    This patch adds an override gflag for consensus metadata fsync so that
    XFS users are less likely to lose their consensus metadata files while
    voting right before a power outage.
    
    Change-Id: I73212d1670a33479cce7d9ef9ee61cfe9b00cdd3
    Reviewed-on: http://gerrit.cloudera.org:8080/12186
    Tested-by: Kudu Jenkins
    Reviewed-by: Adar Dembo <adar@cloudera.com>
    Reviewed-by: Alexey Serbin <aserbin@cloudera.com>
---
 src/kudu/consensus/consensus_meta.cc | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/src/kudu/consensus/consensus_meta.cc b/src/kudu/consensus/consensus_meta.cc
index b075134..355aba9 100644
--- a/src/kudu/consensus/consensus_meta.cc
+++ b/src/kudu/consensus/consensus_meta.cc
@@ -45,6 +45,10 @@ DEFINE_double(fault_crash_before_cmeta_flush, 0.0,
               "consensus metadata. (For testing only!)");
 TAG_FLAG(fault_crash_before_cmeta_flush, unsafe);
 
+DEFINE_bool(cmeta_force_fsync, false,
+            "Whether fsync() should be called when consensus metadata files are updated");
+TAG_FLAG(cmeta_force_fsync, advanced);
+
 namespace kudu {
 namespace consensus {
 
@@ -241,7 +245,12 @@ Status ConsensusMetadata::Flush(FlushMode flush_mode) {
       // essentially an extension of the primary durability mechanism of the
       // consensus subsystem: the WAL. Using the same flag ensures that the WAL
       // and the consensus metadata get the same durability guarantees.
-      FLAGS_log_force_fsync_all ? pb_util::SYNC : pb_util::NO_SYNC),
+      // We add FLAGS_cmeta_force_fsync to support an override in certain
+      // cases. Some filesystems such as ext4 are more forgiving to omitting an
+      // fsync() due to periodic commit with default settings, whereas other
+      // filesystems such as XFS will not commit as often and need the fsync to
+      // avoid significant data loss when a crash happens.
+      FLAGS_log_force_fsync_all || FLAGS_cmeta_force_fsync ? pb_util::SYNC : pb_util::NO_SYNC),
           Substitute("Unable to write consensus meta file for tablet $0 to path $1",
                      tablet_id_, meta_file_path));
   RETURN_NOT_OK(UpdateOnDiskSize());


Mime
View raw message