kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From a...@apache.org
Subject kudu git commit: KUDU-2377: cap GetResourceLimit return value at kint32max
Date Tue, 27 Mar 2018 20:46:31 GMT
Repository: kudu
Updated Branches:
  refs/heads/branch-1.7.x f4383d572 -> 66b827240


KUDU-2377: cap GetResourceLimit return value at kint32max

Resource limits on UNIX are 64-bit unsigned integers but we mistreat them:
1. GetResourceLimit returns the underlying unsigned long as a int64_t.
2. The values may be further truncated into 32-bit integers. For example,
   the LogBlockManager constructor truncates a int64_t value derived from
   the resource limit into a 32-bit integer when constructing its FileCache.

Here's a simple workaround: let's enforce that the returned resource limit
value never exceeds kint32max. Sure, that misrepresents the underlying
system limit, but for the resources we currently define (max open files and
max threads per euid), it's impossible to actually use that much.

I manually tested this by running several unit tests in a root shell with
ulimit -u set to some high values (2^32-1 and 2^32-2).

Change-Id: I0d30ebd5447867d4227974e64786cc48fdbf2688
Reviewed-on: http://gerrit.cloudera.org:8080/9810
Tested-by: Kudu Jenkins
Reviewed-by: Grant Henke <granthenke@apache.org>
Reviewed-by: Todd Lipcon <todd@apache.org>
(cherry picked from commit 17aceda39d486a8f3ed6ddf5ae0e32611154a5ef)
Reviewed-on: http://gerrit.cloudera.org:8080/9821
Tested-by: Adar Dembo <adar@cloudera.com>


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/66b82724
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/66b82724
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/66b82724

Branch: refs/heads/branch-1.7.x
Commit: 66b827240ebcbe9af4c48ab19ad85cbc5774e92a
Parents: f4383d5
Author: Adar Dembo <adar@cloudera.com>
Authored: Mon Mar 26 15:54:29 2018 -0700
Committer: Adar Dembo <adar@cloudera.com>
Committed: Tue Mar 27 20:46:13 2018 +0000

----------------------------------------------------------------------
 src/kudu/util/env.h        | 2 ++
 src/kudu/util/env_posix.cc | 7 ++++++-
 2 files changed, 8 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kudu/blob/66b82724/src/kudu/util/env.h
----------------------------------------------------------------------
diff --git a/src/kudu/util/env.h b/src/kudu/util/env.h
index 4ce3376..279dc18 100644
--- a/src/kudu/util/env.h
+++ b/src/kudu/util/env.h
@@ -321,6 +321,8 @@ class Env {
 
   // Gets the process' current limit for the given resource type.
   //
+  // Returns kint32max if the limit exceeds kint32max or if there is no limit.
+  //
   // On UNIX platforms, this is equivalent to the resource's soft limit.
   virtual int64_t GetResourceLimit(ResourceLimitType t) = 0;
 

http://git-wip-us.apache.org/repos/asf/kudu/blob/66b82724/src/kudu/util/env_posix.cc
----------------------------------------------------------------------
diff --git a/src/kudu/util/env_posix.cc b/src/kudu/util/env_posix.cc
index 2b35805..a8c0c0e 100644
--- a/src/kudu/util/env_posix.cc
+++ b/src/kudu/util/env_posix.cc
@@ -39,6 +39,7 @@
 #include "kudu/gutil/bind.h"
 #include "kudu/gutil/bind_helpers.h"
 #include "kudu/gutil/gscoped_ptr.h"
+#include "kudu/gutil/integral_types.h"
 #include "kudu/gutil/macros.h"
 #include "kudu/gutil/map-util.h"
 #include "kudu/gutil/once.h"
@@ -1588,7 +1589,11 @@ class PosixEnv : public Env {
     // There's no reason for this to ever fail.
     struct rlimit l;
     PCHECK(getrlimit(ResourceLimitTypeToUnixRlimit(t), &l) == 0);
-    return l.rlim_cur;
+
+    // RLIM_INFINITY is defined to -1, which will be 2^64-1 when converted to
+    // rlim_cur's type (uint64_t). Thus it'll also be captured by the
+    // comparison and converted into kint32max.
+    return l.rlim_cur > kint32max ? kint32max : l.rlim_cur;
   }
 
   virtual void IncreaseResourceLimit(ResourceLimitType t) OVERRIDE {


Mime
View raw message