Mailing-List: contact stdcxx-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: stdcxx-dev@incubator.apache.org
Received-SPF: pass (asf.osuosl.org: local policy)
Message-ID: <44F36AE5.4040406@roguewave.com>
Date: Mon, 28 Aug 2006 16:15:01 -0600
From: Andrew Black <ablack@roguewave.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
 rv:1.8.0.5) Gecko/20060720 SeaMonkey/1.0.3
MIME-Version: 1.0
To: stdcxx-dev@incubator.apache.org
Subject: Re: [patch] exec utility child process limits (unix)
References: <44EB7711.30706@roguewave.com> <44EF86D3.10505@roguewave.com>
In-Reply-To: <44EF86D3.10505@roguewave.com>
Content-Type: multipart/mixed;
 boundary="------------010405010103090308000303"

--------------010405010103090308000303
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

Ok.

Attached is take two on the patch.

Probably the major change between the previous version and this one is 
that most things that had been conditionally defined are now 
unconditionally defined.  If a particular limit can't be set on a given 
platform, a warning is issued during option parsing.  I'm not certain 
the warn logic is quite what would be desired, as multiple warnings for 
a given option could be issued if said unsupported option is specified 
multiple times.   Similarly, no warning is provided if the platform 
lacks support for the setrlimit syscall, explaining that all limits will 
fail.

Another failing that I am realizing is that cmdopt.cpp may (will?) fail 
to compile if struct rlimit includes more fields than rlim_cur and 
rlim_max.  Similarly, things may be incorrectly initialized if rlim_max 
is before rlim_cur in the structure.

I've changed things enough that I think a replacement changelog is needed.

--Andrew Black

Log:
     * cmdopt.h [_WIN32 || _WIN64 || !_XOPEN_UNIX]: Define placeholder 
struct rlimit.
       Define struct limit_set, declare child_limits.
     * cmdopt.c [!_WIN32 && !_WIN64 && _XOPEN_UNIX]: Include sys/resource.h
       Define RLIM_INFINITY, RLIM_SAVED_CUR and RLIM_SAVED_MAX if not 
defined, initialize child_limits.
       usage_text[]: Document --ulimit switch.
       parse_limit_opts: Define helper function for parsing --rlimit 
option (borrowed in part from _rw_setopt_ulimit in tests/src/driver.cpp).
       eval_options(): Define opt_ulimit character string, use with 
parse_limit_opts to handle --ulimit command line switch.
     * exec.cpp [!_WIN32 && !_WIN64 && _XOPEN_UNIX]: Include sys/resource.h
       LIMIT: Define helper macro for...
       limit_process(): New helper function to set resource limits, 
based on the values in child_limits (borrowed in part from 
_rw_setopt_ulimit in tests/src/driver.cpp).
       exec_file(): Call above prior to execv.

Martin Sebor wrote:
> Andrew Black wrote:
>> Greetings all.
>>
>> Attached is a patch that enables the exec utility to place limits on 
>> its child process, using the setrlimit() command.  These changes are 
>> based on the _rw_setopt_ulimit function in tests/src/driver.cpp.  It 
>> should be noted that the sys/resource.h header is an XSI extension to 
>> the POSIX standard, so it may not be possible to rely on this header 
>> being present, though an assumption to this effect is made in this 
>> patch and in tests/src/driver.cpp.
> 
> I think either _XOPEN_UNIX or _XOPEN_VERSION is the macro to check
> (after #including <unistd.h>) in order to determine XSI conformance.
> I.e., this would be the magic chant:
> 
>   #include <unistd.h>
> 
>   #if defined (_XOPEN_UNIX)
>       /* XSI extensions supported */
>   #  include <sys/resource.h>
>   #endif   /* _XOPEN_UNIX */
> 
> http://www.opengroup.org/onlinepubs/009695399/basedefs/unistd.h.html
> 
> As for the patch, I'm not sure that recognizing command line options
> only when they are implemented or supported is the best approach. The
> utility contains the (possibly complex) logic to determine whether it
> is or isn't possible to provide the feature so avoiding failures due
> to it being invoked with the unimplemented option will basically mean
> duplicating the same logic in the caller.
> 
> I think handling the unavailability of the options the same was as
> the failure to set the limit (i.e., warning and proceeding) would
> be more robust. After all, on platforms where there is no way to set
> the resource limit we will still want to be able to run our tests.
> 
> Martin

--------------010405010103090308000303
Content-Type: text/x-patch;
 name="ulimit2.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="ulimit2.diff"

Index: exec.cpp
===================================================================
--- exec.cpp	(revision 437752)
+++ exec.cpp	(working copy)
@@ -40,6 +40,9 @@
 #if !defined (_WIN32) && !defined (_WIN64)
 #  include <unistd.h> /* for close, dup, exec, fork */
 #  include <sys/wait.h>
+#  ifdef _XOPEN_UNIX
+#    include <sys/resource.h> /* for setlimit(), RLIMIT_CORE, ... */
+#  endif
 #else
 #  include <windows.h> /* for PROCESS_INFORMATION, ... */
 #  include <process.h> /* for CreateProcess, ... */
@@ -626,6 +629,68 @@
 }
 
 /**
+    Utility macro to generate an rlimit tuple.
+    
+    @parm val 'short' resource name (no leading RLIMIT_).
+    @param idx limit structure name in child_limits global
+    @see limit_process
+    @see child_limits
+*/
+#undef LIMIT
+#define LIMIT(val, idx)   { RLIMIT_ ## val, &child_limits.idx, #val }
+
+/**
+   Set process resource limits, based on the child_limits global.
+
+   This method uses the LIMIT macro to build an internal array of limits
+   to try setting.  If setrlimit fails
+*/
+static void
+limit_process ()
+{
+    static const struct {
+        int resource;
+        struct rlimit* limit;
+        const char* name;
+    } limits[] = {
+#ifdef RLIMIT_CORE
+        LIMIT (CORE, core),
+#endif   // RLIMIT_CORE
+#ifdef RLIMIT_CPU
+        LIMIT (CPU, cpu),
+#endif   // RLIMIT_CPU
+#ifdef RLIMIT_DATA
+        LIMIT (DATA, data),
+#endif   // RLIMIT_DATA
+#ifdef RLIMIT_FSIZE
+        LIMIT (FSIZE, fsize),
+#endif   // RLIMIT_FSIZE
+#ifdef RLIMIT_NOFILE
+        LIMIT (NOFILE, nofile),
+#endif   // RLIMIT_NOFILE
+#ifdef RLIMIT_STACK
+        LIMIT (STACK, stack),
+#endif   // RLIMIT_STACK
+#ifdef RLIMIT_AS
+        LIMIT (AS, as),
+#endif   // RLIMIT_AS    
+        { 0, 0, 0 }
+    };
+
+    for (size_t i = 0; limits [i].limit; ++i) {
+        struct rlimit local;
+
+        memcpy (&local, limits [i].limit, sizeof (struct rlimit));
+
+        if (setrlimit (limits [i].resource, &local)) {
+            warn ("error setting process limits for %s (soft: %lu, hard: "
+                  "%lu): %s\n", limits [i].name, local.rlim_cur, 
+                  local.rlim_max, strerror (errno));
+        }
+    }
+}
+
+/**
    Entry point to the child process (watchdog) subsystem.
 
    This method fork ()s, creating a child process.  This child process becomes
@@ -701,6 +766,8 @@
             terminate (1, "Redirection of stderr to stdout failed: %s\n", 
                        strerror (errno));
 
+        limit_process ();
+
         execv (argv [0], argv);
 
         fprintf (error_file, "%s (%s): execv (\"%s\", ...) error: %s\n",
Index: cmdopt.cpp
===================================================================
--- cmdopt.cpp	(revision 437752)
+++ cmdopt.cpp	(working copy)
@@ -37,6 +37,11 @@
 #include <string.h> /* for str* */
 #if !defined (_WIN32) && !defined (_WIN64)
 #  include <unistd.h> /* for sleep */
+
+#  if defined (_XOPEN_UNIX)
+#    include <sys/resource.h> /* for struct rlimit, RLIMIT_CORE, ... */
+#  endif
+
 #else
 #  include <windows.h> /* for Sleep */
 #endif   /* _WIN{32,64} */
@@ -66,6 +71,28 @@
 const size_t exe_suffix_len = 4; /* strlen(".exe") == 4 */
 #endif
 
+#ifndef RLIM_INFINITY
+#  define RLIM_INFINITY -1
+#endif /* RLIM_INFINITY */
+
+#ifndef RLIM_SAVED_CUR
+#  define RLIM_SAVED_CUR RLIM_INFINITY
+#endif   /* RLIM_SAVED_CUR */
+
+#ifndef RLIM_SAVED_MAX
+#  define RLIM_SAVED_MAX RLIM_INFINITY
+#endif   /* RLIM_SAVED_MAX */
+struct limit_set child_limits = {
+    { RLIM_SAVED_CUR, RLIM_SAVED_MAX },
+    { RLIM_SAVED_CUR, RLIM_SAVED_MAX },
+    { RLIM_SAVED_CUR, RLIM_SAVED_MAX },
+    { RLIM_SAVED_CUR, RLIM_SAVED_MAX },
+    { RLIM_SAVED_CUR, RLIM_SAVED_MAX },
+    { RLIM_SAVED_CUR, RLIM_SAVED_MAX },
+    { RLIM_SAVED_CUR, RLIM_SAVED_MAX },
+    { RLIM_SAVED_CUR, RLIM_SAVED_MAX }
+};
+
 static const char
 usage_text[] = {
     "Usage: %s [OPTIONS] [targets]\n"
@@ -91,12 +118,39 @@
     "    --sleep=sec  Sleep for the specified number of seconds.\n"
     "    --signal=sig Send itself the specified signal.\n"
     "    --ignore=sig Ignore the specified signal.\n"
+    "    --ulimit=lim Set child process usage limits (see below).\n"
     "\n"
     "  All short (single dash) options must be specified seperately.\n"
     "  If a short option takes a value, it may either be provided like\n"
     "  '-sval' or '-s val'.\n"
     "  If a long option take a value, it may either be provided like\n"
     "  '--option=value' or '--option value'.\n"
+    "\n"
+    "  --ulimit sets limits on how much of a given resource or resorces\n"
+    "  child processes are allowed to utilize.  These limits take on two\n"
+    "  forms, 'soft' and 'hard' limits.  Options are specified in the form\n"
+    "  'resource:limit', where resource is a resource named below, and and\n"
+    "  limit is a number, with value specifing the limit for the named\n"
+    "  resource.  If multiple limits are to be set, they can be specified\n"
+    "  either in multiple instances of the --ulimit switch, or by specifying\n"
+    "  additional limits in the same call by seperating the pairs with\n"
+    "  commas.  'Soft' limits are specified by providing the resource name\n"
+    "  in lowercase letters, while 'hard' limits are specified by providing\n"
+    "  the resource name in uppercase letters.  To set both limits, specify\n"
+    "  the resource name in title case.\n"
+    "\n"
+    "  --ulimit modes:\n"
+    "    core   Maximum size of core file, in bytes.\n"
+    "    cpu    Maximum CPU time, in seconds.\n"
+    "    data   Maximum data segment size, in bytes.\n"
+    "    fsize  Maximum size of generated files, in bytes.\n"
+    "    nofile Maximum number of open file descriptors.\n"
+    "    stack  Maximum size of initial thread's stack, in bytes.\n"
+    "    as     Maximum size of available memory, in bytes.\n"
+    "\n"
+    "  Note: Some operating systems lack support for some or all of the\n"
+    "  ulimit modes.  If a system is unable to limit a given property, a\n"
+    "  warning message will be produced.\n"
 };
 
 #if !defined (_WIN32) && !defined (_WIN64)
@@ -201,6 +255,128 @@
 }
 
 /**
+   Helper function to parse a ulimit value string
+
+   @param opts ulimit value string to pares
+   @see child_limits
+*/
+static bool
+parse_limit_opts (const char* opts)
+{
+    static const struct {
+        struct rlimit* limit;
+        const char* name;
+        const char* caps;
+        const char* mixd;
+        size_t len;
+    } limits[] = {
+        {
+#ifdef RLIMIT_CORE
+            &child_limits.core,
+#else
+            0,
+#endif   // RLIMIT_CORE
+            "core", "CORE", "Core", 4 },
+        {
+#ifdef RLIMIT_CPU
+            &child_limits.cpu,
+#else
+            0,
+#endif   // RLIMIT_CPU
+            "cpu", "CPU", "Cpu", 3 },
+        {
+#ifdef RLIMIT_DATA
+            &child_limits.data,
+#else
+            0,
+#endif   // RLIMIT_DATA
+            "data", "DATA", "Data", 4 },
+        {
+#ifdef RLIMIT_FSIZE
+            &child_limits.fsize,
+#else
+            0,
+#endif   // RLIMIT_FSIZE
+            "fsize", "FSIZE", "Fsize", 5 },
+        {
+#ifdef RLIMIT_NOFILE
+            &child_limits.nofile,
+#else
+            0,
+#endif   // RLIMIT_NOFILE
+            "nofile", "NOFILE", "Nofile", 6 },
+        {
+#ifdef RLIMIT_STACK
+            &child_limits.stack,
+#else
+            0,
+#endif   // RLIMIT_STACK
+            "stack", "STACK", "Stack", 5 },
+        {
+#ifdef RLIMIT_AS
+            &child_limits.as,
+#else
+            0,
+#endif   // RLIMIT_AS    
+            "as", "AS", "As", 2 },
+        { 0, 0, 0, 0, 0 }
+    };
+
+    const char* arg = opts;
+
+    assert (0 != opts);
+
+    while (arg && *arg) {
+
+        const size_t arglen = strlen (arg);
+
+        for (size_t i = 0; limits [i].name; ++i) {
+            if (   limits [i].len < arglen
+                && (   0 == memcmp (limits [i].name, arg, limits [i].len)
+                    || 0 == memcmp (limits [i].caps, arg, limits [i].len)
+                    || 0 == memcmp (limits [i].mixd, arg, limits [i].len))
+                && ':' == arg [limits [i].len]) {
+
+                // determine whether the hard limit and/or
+                // the soft limit should be set
+                const bool hard = isupper (arg [0]);
+                const bool soft = islower (arg [1]);
+
+                arg += limits [i].len + 1;
+
+                char *end;
+                const long lim = strtol (arg, &end, 10);
+
+                arg = end;
+
+                if ('\0' != *arg && ',' != *arg)
+                    break;
+
+                if (limits [i].limit) {
+                    if (soft)
+                        limits [i].limit->rlim_cur = lim;
+
+                    if (hard)
+                        limits [i].limit->rlim_max = lim;
+                } else
+                    warn ("Unable to process %s limit: Not supported\n");
+
+                break;
+            }
+        }
+
+        if (',' == *arg) {
+            ++arg;
+        }
+        else if ('\0' != *arg) {
+            return 1;
+        }
+    }
+    return 0;
+}
+
+
+/**
     Helper function to produce 'Bad argument' error message.
     
     @param opt name of option encountered
@@ -266,6 +442,7 @@
     const char opt_nocompat[] = "--nocompat";
     const char opt_signal[]   = "--signal";
     const char opt_sleep[]    = "--sleep";
+    const char opt_ulimit[]   = "--ulimit";
 
     int i;
 
@@ -397,7 +574,16 @@
                     }
                 }
             }
-
+            else if (   sizeof opt_ulimit <= arglen
+                     && !memcmp (opt_ulimit, argv [i], sizeof opt_ulimit - 1)) {
+                optname = opt_ulimit;
+                optarg  = get_long_val (argv, &i, sizeof opt_ulimit);
+                if (optarg && *optarg) {
+                    if (!parse_limit_opts (optarg)) {
+                        break;
+                    }
+                }
+            }
             /* fall through */
         }
         default:
Index: cmdopt.h
===================================================================
--- cmdopt.h	(revision 437752)
+++ cmdopt.h	(working copy)
@@ -39,6 +39,27 @@
 extern const char suffix_sep; /**< File suffix seperator. */
 extern const size_t exe_suffix_len; /**< Length of executable suffix. */
 
+#if defined (_WIN32) || defined (_WIN64) || !defined (_XOPEN_UNIX)
+/**
+   Placeholder rlimit structure for platforms it isn't defined on.
+*/
+struct rlimit {
+    long rlim_cur;
+    long rlim_max;
+};
+#endif
+
+extern struct limit_set {
+    struct rlimit core;
+    struct rlimit cpu;
+    struct rlimit data;
+    struct rlimit fsize;
+    struct rlimit nofile;
+    struct rlimit stack;
+    struct rlimit mem;
+    struct rlimit as;
+} child_limits; /**< Container holding child process limits. */
+
 void 
 show_usage (int status);
 

--------------010405010103090308000303--