aurora-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mchucarr...@apache.org
Subject git commit: Stage 1 of implementing command hooks for aurora v2.
Date Tue, 29 Apr 2014 19:04:32 GMT
Repository: incubator-aurora
Updated Branches:
  refs/heads/master 519b5871c -> 891127320


Stage 1 of implementing command hooks for aurora v2.

This change includes:
(1) The ability to add hard-wired hooks, by registering them in ConfigurationPlugins
  compiled into a pex;
(2) Dynamically loaded plugins, loaded from plugin files.

The dynamically loaded plugins are *not* currently active outside of tests.

The second stage of this change will activate dynamically loaded plugins, and
provide a mechanism to allow privileged users to override hooks.

Bugs closed: aurora-270

Reviewed at https://reviews.apache.org/r/20687/


Project: http://git-wip-us.apache.org/repos/asf/incubator-aurora/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-aurora/commit/89112732
Tree: http://git-wip-us.apache.org/repos/asf/incubator-aurora/tree/89112732
Diff: http://git-wip-us.apache.org/repos/asf/incubator-aurora/diff/89112732

Branch: refs/heads/master
Commit: 891127320e6aa11f44610ea3e7ed923841544b89
Parents: 519b587
Author: Mark Chu-Carroll <mchucarroll@twopensource.com>
Authored: Tue Apr 29 15:01:51 2014 -0400
Committer: Mark Chu-Carroll <mchucarroll@twitter.com>
Committed: Tue Apr 29 15:01:51 2014 -0400

----------------------------------------------------------------------
 docs/design/command-hooks.md                    | 209 +++++++++++++++++++
 src/main/python/apache/aurora/client/cli/BUILD  |   1 +
 .../python/apache/aurora/client/cli/__init__.py |  36 +++-
 .../apache/aurora/client/cli/command_hooks.py   | 154 ++++++++++++++
 .../python/apache/aurora/client/cli/jobs.py     |   2 +-
 .../python/apache/aurora/client/cli/AuroraHooks |  29 +++
 src/test/python/apache/aurora/client/cli/BUILD  |  14 ++
 .../cli/hook_test_data/bad_syntax/AuroraHooks   |   1 +
 .../cli/hook_test_data/exec_error/AuroraHooks   |  30 +++
 .../aurora/client/cli/test_command_hooks.py     | 198 ++++++++++++++++++
 10 files changed, 671 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/docs/design/command-hooks.md
----------------------------------------------------------------------
diff --git a/docs/design/command-hooks.md b/docs/design/command-hooks.md
new file mode 100644
index 0000000..ee320ed
--- /dev/null
+++ b/docs/design/command-hooks.md
@@ -0,0 +1,209 @@
+# Command Hooks for the Aurora Client
+
+## Introduction/Motivation
+
+We've got hooks in the client that surround API calls. These are
+pretty awkward, because they don't correlate with user actions. For
+example, suppose we wanted a policy that said users weren't allowed to
+kill all instances of a production job at once.
+
+Right now, all that we could hook would be the "killJob" api call. But
+kill (at least in newer versions of the client) normally runs in
+batches. If a user called killall, what we would see on the API level
+is a series of "killJob" calls, each of which specified a batch of
+instances. We woudn't be able to distinguish between really killing
+all instances of a job (which is forbidden under this policy), and
+carefully killing in batches (which is permitted.) In each case, the
+hook would just see a series of API calls, and couldn't find out what
+the actual command being executed was!
+
+For most policy enforcement, what we really want to be able to do is
+look at and vet the commands that a user is performing, not the API
+calls that the client uses to implement those commands.
+
+So I propose that we add a new kind of hooks, which surround noun/verb
+commands. A hook will register itself to handle a collection of (noun,
+verb) pairs. Whenever any of those noun/verb commands are invoked, the
+hooks methods will be called around the execution of the verb. A
+pre-hook will have the ability to reject a command, preventing the
+verb from being executed.
+
+## Registering Hooks
+
+These hooks will be registered two ways:
+* Project hooks file. If a file named `AuroraHooks` is in the project directory
+  where an aurora command is being executed, that file will be read,
+  and its hooks will be registered.
+* Configuration plugins. A configuration plugin can register hooks using an API.
+  Hooks registered this way are, effectively, hardwired into the client executable.
+
+The order of execution of hooks is unspecified: they may be called in
+any order. There is no way to guarantee that one hook will execute
+before some other hook.
+
+
+### Global Hooks
+
+Commands registered by the python call are called _global_ hooks,
+because they will run for all configurations, whether or not they
+specify any hooks in the configuration file.
+
+In the implementation, hooks are registered in the module
+`apache.aurora.client.cli.command_hooks`, using the class `GlobalCommandHookRegistry`.  A
+global hook can be registered by calling `GlobalCommandHookRegistry.register_command_hook`
+in a configuration plugin.
+
+### Hook Files
+
+A hook file is a file containing Python source code. It will be
+dynamically loaded by the Aurora command line executable. After
+loading, the client will check the module for a global variable named
+"hooks", which contains a list of hook objects, which will be added to
+the hook registry.
+
+A project hooks file will be named `AuroraHooks`,
+and will be located in either the directory where the command is being
+executed, or one of its parent directories, up to the nearest git/mercurial
+repository base.
+
+### The API
+
+    class CommandHook(object)
+      @property
+      def name(self):
+        """Returns a name for the hook."
+
+      def get_nouns(self):
+        """Return the nouns that have verbs that should invoke this hook."""
+
+      def get_verbs(self, noun):
+        """Return the verbs for a particular noun that should invoke his hook."""
+
+      @abstractmethod
+      def pre_command(self, noun, verb, context, commandline):
+        """Execute a hook before invoking a verb.
+        * noun: the noun being invoked.
+        * verb: the verb being invoked.
+        * context: the context object that will be used to invoke the verb.
+          The options object will be initialized before calling the hook
+        * commandline: the original argv collection used to invoke the client.
+        Returns: True if the command should be allowed to proceed; False if the command
+        should be rejected.
+        """
+
+      def post_command(self, noun, verb, context, commandline, result):
+        """Execute a hook after invoking a verb.
+        * noun: the noun being invoked.
+        * verb: the verb being invoked.
+        * context: the context object that will be used to invoke the verb.
+          The options object will be initialized before calling the hook
+        * commandline: the original argv collection used to invoke the client.
+        * result: the result code returned by the verb.
+        Returns: nothing
+        """
+
+    class GlobalCommandHookRegistry(object):
+      @classmethod
+      def register_command_hook(self, hook):
+        pass
+
+## Skipping Hooks
+
+In a perfect world, hooks would represent a global property or policy
+that should always be enforced. Unfortunately, we don't live in a
+perfect world, which means that sometimes, every rule needs to get
+broken.
+
+For example, an organization could decide that every configuration
+must be checked in to source control before it could be
+deployed. That's an entirely reasonable policy. It would be easy to
+implement it using a hook. But what if there's a problem, and the
+source repos is down?
+
+The easiest solution is just to allow a user to add a `--skip-hooks`
+flag to the command-line. But doing that means that an organization
+can't actually use hooks to enforce policy, because users can skip
+them whenever they want.
+
+Instead, we'd like to have a system where it's possible to create
+hooks to enforce policy, and then include a way of building policy
+about when hooks can be skipped.
+
+I'm using sudo as a rough model for this. Many organizations need to
+give people the ability to run privileged commands, but they still
+want to have some control. Sudo allows them to specify who is allowed
+to run a privileged command; where they're allowed to run it; and what
+command(s) they're allowed to run.  All of that is specified in a
+special system file located in `/etc/sudoers` on a typical unix
+machine.
+
+### Specifying when hooks can be skipped
+
+The sudoers file has a terrible syntax, so I'm not going to try to
+adopt it; instead, I'm going to stick with the Pystachio-based
+configuration syntax that we use in Aurora. A rule that permits a
+group of users to skip hooks is defined using a Pystachio struct:
+
+    class HookRule(Struct):
+      roles = List(String)
+      commands = Map(String, List(String))
+      arg_patterns = List(String)
+	  hooks = List(String)
+
+* `roles` is a list of role names, or regular expressions that range over role
+  names. This rule gives permission to those users to skip hooks.
+* `commands` is a map from nouns to lists of verbs. If a command `aurora n v`
+  is being executed, this rule allows the hooks to be skipped if
+  `v` is in `commands[n]`. If this is empty, then all commands can be skipped.
+* `arg_patterns` is a list of regular expressions ranging over parameters.
+  If any of the parameters of the command match the parameters in this list,
+  the hook can be skipped.
+* `hooks` is a list of hook identifiers which can be skipped by a user
+  that satisfies this rule.
+
+The hooks file defines a global variable `hook_rules`, which is a list of
+`HookRule` objects. If any of the hook rules matches, then the command
+can be run with hooks skipped.
+
+For example, the following is a hook rules file which allows:
+* The admin (role admin) to skip any hook.
+* Any user to skip hooks for test jobs.
+* A specific group of users to skip hooks for jobs in cluster `east`
+* Another group of users to skip hooks for `job kill` in cluster `west`.
+
+    allow_admin = HookRule(roles=['admin'])
+    allow_test = HookRule(roles=['.*'],  arg_patterns=['.*/.*/test/.*'])
+    allow_east_users = HookRule(roles=['john', 'mary', 'mike', 'sue'],
+        arg_patterns=['east/.*/.*./*'])
+    allow_west_kills = HookRule(roles=['anne', 'bill', 'chris'],
+      commands = { 'job': ['kill']}, arg_patterns = ['west/.*/.*./*'])
+
+    hook_rules = [allow_admin, allow_test, allow_east_users, allow_west_kills]
+
+## Skipping Hooks
+
+To skip a hook, a user uses a command-line option, `--skip-hooks`. The option can either
+specify specific hooks to skip, or "all":
+
+* `aurora --skip-hooks=all job create east/bozo/devel/myjob` will create a job
+  without running any hooks.
+* `aurora --skip-hooks=test,iq create east/bozo/devel/myjob` will create a job,
+  and will skip only the hooks named "test" and "iq".
+
+
+## Changes
+
+Major changes between this and the last version of this proposal.
+* Command hooks can't be declared in a configuration file. There's a simple
+  reason why: hooks run before a command's implementation is invoked.
+  Config files are read during the commands invocation if necessary. If the
+  hook is declared in the config file, by the time you know that it should
+  have been run, it's too late. So I've removed config-hooks from the
+  proposal. (API hooks defined by configs still work.)
+* Skipping hooks. We expect aurora to be used inside of large
+  organizations. One of the primary use-cases of hooks is to create
+  enforcable policy that are specific to an organization. If hooks
+  can just be skipped because a user wants to skip them, it means that
+  the policy can't be enforced, which defeats the purpose of having them.
+  So in this update, I propose a mechanism, loosely based on a `sudo`-like
+  mechanism for defining when hooks can be skipped.

http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/main/python/apache/aurora/client/cli/BUILD
----------------------------------------------------------------------
diff --git a/src/main/python/apache/aurora/client/cli/BUILD b/src/main/python/apache/aurora/client/cli/BUILD
index 17cdc28..1bd565e 100644
--- a/src/main/python/apache/aurora/client/cli/BUILD
+++ b/src/main/python/apache/aurora/client/cli/BUILD
@@ -43,6 +43,7 @@ python_library(
   sources = [
     '__init__.py',
     'context.py',
+    'command_hooks.py',
     'jobs.py',
     'options.py',
     'quota.py',

http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/main/python/apache/aurora/client/cli/__init__.py
----------------------------------------------------------------------
diff --git a/src/main/python/apache/aurora/client/cli/__init__.py b/src/main/python/apache/aurora/client/cli/__init__.py
index 78c109d..1e396b2 100644
--- a/src/main/python/apache/aurora/client/cli/__init__.py
+++ b/src/main/python/apache/aurora/client/cli/__init__.py
@@ -40,6 +40,7 @@ import sys
 from uuid import uuid1
 
 from twitter.common.python.pex import PexInfo
+from .command_hooks import GlobalCommandHookRegistry
 
 # Constants for standard return codes.
 EXIT_OK = 0
@@ -308,6 +309,30 @@ class CommandLine(object):
       self.register_nouns()
     return self.nouns.keys()
 
+  def run_pre_hooks(self, context, noun, verb, args, command_hooks):
+    try:
+      for hook in command_hooks:
+        result = hook.pre_command(noun, verb, context, args)
+        if result != 0:
+          print_aurora_log(logging.INFO, 'Command hook %s aborted operation with error code
%s' %
+              (hook.name, result))
+          self.print_out('Command aborted by command hook %s' % hook.name)
+          return result
+      return EXIT_OK
+    except (Context.CommandError, ConfigurationPlugin.Error) as c:
+      print_aurora_log(logging.INFO, 'Error executing command hook %s: %s' % (hook.name,
c))
+      self.print_err('Error executing command hook %s: %s; aborting' % hook.name, c.msg)
+      return c.code
+
+  def run_post_hooks(self, context, noun, verb, args, result, command_hooks):
+    try:
+      for hook in command_hooks:
+        hook.post_command(noun, verb, context, args, result)
+    except (Context.CommandError, ConfigurationPlugin.Error) as c:
+      print_aurora_log(logging.INFO, 'Error executing post-command hook %s: %s' % (hook.name,
c))
+      self.print_err('Error executing command hook %s: %s; aborting' % hook.name, c.msg)
+      return c.code
+
   def execute(self, args):
     """Execute a command.
     :param args: the command-line arguments for the command. This only includes arguments
@@ -337,21 +362,28 @@ class CommandLine(object):
     except ConfigurationPlugin.Error as e:
       print('Error in configuration plugin before execution: %s' % c.msg, file=sys.stderr)
       return c.code
+    command_hooks = GlobalCommandHookRegistry.get_command_hooks_for(options.noun, options.verb)
+    plugin_result = self.run_pre_hooks(context, options.noun, options.verb, args, command_hooks)
+    if plugin_result != 0:
+      return plugin_result
+
     try:
       result = noun.execute(context)
       if result == EXIT_OK:
         print_aurora_log(logging.INFO, 'Command terminated successfully')
+        self.run_post_hooks(options.noun, options.verb, context, args, result, command_hooks)
       else:
         print_aurora_log(logging.INFO, 'Commmand terminated with error code %s', result)
+
       for plugin in self.plugins:
         try:
           plugin.after_execution(context, result)
         except ConfigurationPlugin.Error as e:
-          print_aurora_log(logging.INFO, 'Error executing post-execution extension hook:
 %s', e.msg)
+          print_aurora_log(logging.INFO, 'Error executing post-execution plugin: %s', e.msg)
       return result
     except Context.CommandError as c:
       print_aurora_log(logging.INFO, 'Error executing command: %s', c.msg)
-      print('Error executing command: %s' % c.msg, file=sys.stderr)
+      self.print_err('Error executing command: %s' % c.msg)
       return c.code
 
 

http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/main/python/apache/aurora/client/cli/command_hooks.py
----------------------------------------------------------------------
diff --git a/src/main/python/apache/aurora/client/cli/command_hooks.py b/src/main/python/apache/aurora/client/cli/command_hooks.py
new file mode 100644
index 0000000..2d20068
--- /dev/null
+++ b/src/main/python/apache/aurora/client/cli/command_hooks.py
@@ -0,0 +1,154 @@
+#
+# Copyright 2014 Apache Software Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from __future__ import print_function
+
+from abc import abstractmethod
+import logging
+import os
+import sys
+
+from twitter.common.lang import Compatibility
+
+
+class GlobalCommandHookRegistry(object):
+  """Registry for command hooks.
+  See docs/design/command-hooks.md for details on what hooks are, and how
+  they work.
+  """
+
+  COMMAND_HOOKS = []
+  HOOKS_FILE_NAME = 'AuroraHooks'
+
+  @classmethod
+  def load_hooks_file(cls, path):
+    """Load a file containing hooks. If there are any errors compiling or executing the file,
+    the errors will be logged, the hooks from the file will be skipped, but the execution
of
+    the command will continue.
+    """
+    #  To load a hooks file, we compile and exec the file.
+    with open(path, "r") as hooks_file:
+      hooks_data = hooks_file.read()
+      hooks_code = None
+      try:
+        hooks_code = compile(hooks_data, path, "exec")
+      except (SyntaxError, TypeError) as e:
+        logging.warn("Error compiling hooks file %s: %s" % (path, e))
+        print("Error compiling hooks file %s: %s" % (path, e), file=sys.stderr)
+        return {}
+      hooks_environment = {}
+      try:
+        Compatibility.exec_function(hooks_code, hooks_environment)
+      except Exception as e:
+        # Unfortunately, exec could throw *anything* at all.
+        logging.warn('Warning: error loading hooks file %s: %s' % (path, e))
+        print('Warning: error loading hooks file %s: %s' % (path, e), file=sys.stderr)
+        return {}
+      for hook in hooks_environment.get('hooks', []):
+        cls.register_command_hook(hook)
+      return hooks_environment
+
+  @classmethod
+  def find_project_hooks_file(self, dir):
+    """Search for a file named "AuroraHooks" in  current directory or
+    one of its parents, up to the closest repository root. Only one
+    file will be loaded, so creating an AuroraHooks file in a subdirecory will
+    override and replace the one in the parent directory
+    """
+    def is_repos_root(dir):
+      # a directory is a git root if it contains a directory named ".git".
+      # it's an HG root if it contains a directory named ".hg"
+      return any(os.path.isdir(os.path.join(dir, rootname)) for rootname in ['.git', '.hg'])
+
+    filepath =  os.path.join(dir, self.HOOKS_FILE_NAME)
+    if os.path.exists(filepath):
+      return filepath
+    elif is_repos_root(dir):
+      # This is a repository root, so stop looking.
+      return None
+    else:
+      parent, _ = os.path.split(os.path.abspath(dir))
+      # if we've reached the filesystem root, then parent==self.
+      if parent == dir:
+        return None
+      else:
+        return find_project_hooks_file(parent)
+      return None
+
+  @classmethod
+  def load_project_hooks(cls, dir="."):
+    """Looks for a project hooks file. Project hooks will be in the current directory,
+    or one of its parents. Stops looking at the root of an SCM repository.
+    """
+    hooks_file = cls.find_project_hooks_file(dir)
+    if hooks_file is not None:
+      return cls.load_hooks_file(hooks_file)
+    else:
+      return None
+
+  @classmethod
+  def reset(cls):
+    """For testing purposes, reset the list of registered hooks"""
+    cls.COMMAND_HOOKS = []
+
+  @classmethod
+  def get_command_hooks_for(cls, noun, verb):
+    """Looks up the set of hooks tht should apply for a (noun, verb) pair.
+    """
+    return [hook for hook in cls.COMMAND_HOOKS if noun in hook.get_nouns() and
+        verb in hook.get_verbs(noun)]
+
+  @classmethod
+  def register_command_hook(cls, hook):
+    cls.COMMAND_HOOKS.append(hook)
+
+
+class CommandHook(object):
+  @property
+  def name(self):
+    return None
+
+  @abstractmethod
+  def get_nouns(self):
+    """Return the nouns that have verbs that should invoke this hook."""
+
+  @abstractmethod
+  def get_verbs(self, noun):
+    """Return the verbs for a particular noun that should invoke his hook."""
+
+  @abstractmethod
+  def pre_command(self, noun, verb, context, commandline):
+    """Execute a hook before invoking a verb.
+    * noun: the noun being invoked.
+    * verb: the verb being invoked.
+    * context: the context object that will be used to invoke the verb.
+         The options object will be initialized before calling the hook
+    * commandline: the original argv collection used to invoke the client.
+    Returns: 0 if the command should be allowed to proceed; otherwise, the exit code
+       to return at the command line.
+    """
+
+  @abstractmethod
+  def post_command(self, noun, verb, context, commandline, result):
+    """Execute a hook after invoking a verb.
+    * noun: the noun being invoked.
+    * verb: the verb being invoked.
+    * context: the context object that will be used to invoke the verb.
+    * commandline: the original argv collection used to invoke the client.
+    * result: the result code returned by the verb.
+    Returns: nothing
+    """
+

http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/main/python/apache/aurora/client/cli/jobs.py
----------------------------------------------------------------------
diff --git a/src/main/python/apache/aurora/client/cli/jobs.py b/src/main/python/apache/aurora/client/cli/jobs.py
index bd7a308..782b348 100644
--- a/src/main/python/apache/aurora/client/cli/jobs.py
+++ b/src/main/python/apache/aurora/client/cli/jobs.py
@@ -304,7 +304,7 @@ class AbstractKillCommand(Verb):
         batch.append(instances_to_kill.pop())
       resp = api.kill_job(job, batch)
       if resp.responseCode is not ResponseCode.OK:
-        resp.print_log('Kill of shards %s failed with error %s' % (batch, resp.message))
+        context.print_log('Kill of shards %s failed with error %s' % (batch, resp.message))
         errors += 1
         if errors > context.options.max_total_failures:
           raise context.CommandError(EXIT_COMMAND_FAILURE,

http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/test/python/apache/aurora/client/cli/AuroraHooks
----------------------------------------------------------------------
diff --git a/src/test/python/apache/aurora/client/cli/AuroraHooks b/src/test/python/apache/aurora/client/cli/AuroraHooks
new file mode 100644
index 0000000..c25bcfb
--- /dev/null
+++ b/src/test/python/apache/aurora/client/cli/AuroraHooks
@@ -0,0 +1,29 @@
+from apache.aurora.client.cli.command_hooks import CommandHook
+
+class DynamicHook(CommandHook):
+
+  def __init__(self, succeed):
+    self.succeed = succeed
+    self.ran_pre = False
+    self.ran_post = False
+
+  def get_nouns(self):
+    return ['job']
+
+  def get_verbs(self, noun):
+    if noun == 'job':
+      return ['create', 'status' ]
+    else:
+      return []
+
+  def pre_command(self, noun, verb, context, commandline):
+    self.ran_pre = True
+    if self.succeed:
+      return 0
+    else:
+      return 1
+
+  def post_command(self, noun, verb, context, commandline, result):
+    self.ran_post = True
+
+hooks = [ DynamicHook(True) ]
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/test/python/apache/aurora/client/cli/BUILD
----------------------------------------------------------------------
diff --git a/src/test/python/apache/aurora/client/cli/BUILD b/src/test/python/apache/aurora/client/cli/BUILD
index 34fdb47..1504476 100644
--- a/src/test/python/apache/aurora/client/cli/BUILD
+++ b/src/test/python/apache/aurora/client/cli/BUILD
@@ -18,6 +18,7 @@ python_test_suite(
   name = 'all',
   dependencies = [
     pants(':bridge'),
+    pants(':command_hooks'),
     pants(':help'),
     pants(':job'),
     pants(':logging'),
@@ -49,6 +50,19 @@ python_tests(
 )
 
 python_tests(
+  name = 'command_hooks',
+  sources = [ 'test_command_hooks.py' ],
+  dependencies = [
+    pants(':util'),
+    pants('3rdparty/python:mock'),
+    pants('3rdparty/python:twitter.common.contextutil'),
+    pants('src/main/python/apache/aurora/client/cli'),
+    pants('src/main/python/apache/aurora/client/cli:client'),
+    pants('src/test/python/apache/aurora/client/commands:util')
+  ]
+)
+
+python_tests(
   name = 'logging',
   sources = [ 'test_logging.py' ],
   dependencies = [

http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/test/python/apache/aurora/client/cli/hook_test_data/bad_syntax/AuroraHooks
----------------------------------------------------------------------
diff --git a/src/test/python/apache/aurora/client/cli/hook_test_data/bad_syntax/AuroraHooks
b/src/test/python/apache/aurora/client/cli/hook_test_data/bad_syntax/AuroraHooks
new file mode 100644
index 0000000..6333986
--- /dev/null
+++ b/src/test/python/apache/aurora/client/cli/hook_test_data/bad_syntax/AuroraHooks
@@ -0,0 +1 @@
+This is not valid python syntax.

http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/test/python/apache/aurora/client/cli/hook_test_data/exec_error/AuroraHooks
----------------------------------------------------------------------
diff --git a/src/test/python/apache/aurora/client/cli/hook_test_data/exec_error/AuroraHooks
b/src/test/python/apache/aurora/client/cli/hook_test_data/exec_error/AuroraHooks
new file mode 100644
index 0000000..768c124
--- /dev/null
+++ b/src/test/python/apache/aurora/client/cli/hook_test_data/exec_error/AuroraHooks
@@ -0,0 +1,30 @@
+from apache.aurora.client.cli.command_hooks import CommandHook
+
+class DynamicHook(CommandHook):
+
+  def __init__(self, succeed):
+    self.succeed = succeed
+    self.ran_pre = False
+    self.ran_post = False
+
+  def get_nouns(self):
+    return ['job']
+
+  def get_verbs(self, noun):
+    if noun == 'job':
+      return ['create', 'status' ]
+    else:
+      return []
+
+  def pre_command(self, noun, verb, context, commandline):
+    self.ran_pre = True
+    if self.succeed:
+      return 0
+    else:
+      return 1
+
+  def post_command(self, noun, verb, context, commandline, result):
+    self.ran_post = True
+
+
+hooks = [ DynamicHook(True), 1/0 ]

http://git-wip-us.apache.org/repos/asf/incubator-aurora/blob/89112732/src/test/python/apache/aurora/client/cli/test_command_hooks.py
----------------------------------------------------------------------
diff --git a/src/test/python/apache/aurora/client/cli/test_command_hooks.py b/src/test/python/apache/aurora/client/cli/test_command_hooks.py
new file mode 100644
index 0000000..7c6f70c
--- /dev/null
+++ b/src/test/python/apache/aurora/client/cli/test_command_hooks.py
@@ -0,0 +1,198 @@
+#
+# Copyright 2014 Apache Software Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from twitter.common.contextutil import temporary_file
+
+from gen.apache.aurora.api.ttypes import (
+    AssignedTask,
+    Identity,
+    ScheduledTask,
+    ScheduleStatus,
+    ScheduleStatusResult,
+    TaskEvent,
+    TaskQuery,
+)
+
+from apache.aurora.client.cli.client import AuroraCommandLine
+from apache.aurora.client.cli.command_hooks import CommandHook, GlobalCommandHookRegistry
+from apache.aurora.client.cli.util import AuroraClientCommandTest, FakeAuroraCommandContext
+from apache.aurora.config import AuroraConfig
+
+from mock import Mock, patch
+
+
+class HookForTesting(CommandHook):
+  def __init__(self, succeed):
+    self.succeed = succeed
+    self.ran_pre = False
+    self.ran_post = False
+
+  def get_nouns(self):
+    return ['job']
+
+  def get_verbs(self, noun):
+    if noun == 'job':
+      return ['create', 'status']
+    else:
+      return []
+
+  def pre_command(self, noun, verb, context, commandline):
+    self.ran_pre = True
+    if self.succeed:
+      return 0
+    else:
+      return 1
+
+  def post_command(self, noun, verb, context, commandline, result):
+    self.ran_post = True
+
+
+class TestClientCreateCommand(AuroraClientCommandTest):
+
+  @classmethod
+  def create_mock_task(cls, task_id, instance_id, initial_time, status):
+    mock_task = Mock(spec=ScheduledTask)
+    mock_task.assignedTask = Mock(spec=AssignedTask)
+    mock_task.assignedTask.taskId = task_id
+    mock_task.assignedTask.instanceId = instance_id
+    mock_task.status = status
+    mock_task_event = Mock(spec=TaskEvent)
+    mock_task_event.timestamp = initial_time
+    mock_task.taskEvents = [mock_task_event]
+    return mock_task
+
+  @classmethod
+  def create_mock_status_query_result(cls, scheduleStatus):
+    mock_query_result = cls.create_simple_success_response()
+    mock_query_result.result.scheduleStatusResult = Mock(spec=ScheduleStatusResult)
+    if scheduleStatus == ScheduleStatus.INIT:
+      # status query result for before job is launched.
+      mock_query_result.result.scheduleStatusResult.tasks = []
+    else:
+      mock_task_one = cls.create_mock_task('hello', 0, 1000, scheduleStatus)
+      mock_task_two = cls.create_mock_task('hello', 1, 1004, scheduleStatus)
+      mock_query_result.result.scheduleStatusResult.tasks = [mock_task_one, mock_task_two]
+    return mock_query_result
+
+  @classmethod
+  def create_mock_query(cls):
+    return TaskQuery(owner=Identity(role=cls.TEST_ROLE), environment=cls.TEST_ENV,
+        jobName=cls.TEST_JOB)
+
+  @classmethod
+  def get_createjob_response(cls):
+    # Then, we call api.create_job(config)
+    return cls.create_simple_success_response()
+
+  @classmethod
+  def get_failed_createjob_response(cls):
+    return cls.create_error_response()
+
+  @classmethod
+  def assert_create_job_called(cls, mock_api):
+    # Check that create_job was called exactly once, with an AuroraConfig parameter.
+    assert mock_api.create_job.call_count == 1
+    assert isinstance(mock_api.create_job.call_args_list[0][0][0], AuroraConfig)
+
+  @classmethod
+  def assert_scheduler_called(cls, mock_api, mock_query, num_queries):
+    assert mock_api.scheduler_proxy.getTasksStatus.call_count == num_queries
+    mock_api.scheduler_proxy.getTasksStatus.assert_called_with(mock_query)
+
+  def test_create_job_with_successful_hook(self):
+    GlobalCommandHookRegistry.reset()
+    command_hook = HookForTesting(True)
+    GlobalCommandHookRegistry.register_command_hook(command_hook)
+    self.generic_test_successful_hook(command_hook)
+
+  def generic_test_successful_hook(self, command_hook):
+    mock_context = FakeAuroraCommandContext()
+    with patch('apache.aurora.client.cli.jobs.Job.create_context', return_value=mock_context):
+      mock_query = self.create_mock_query()
+      mock_context.add_expected_status_query_result(
+        self.create_mock_status_query_result(ScheduleStatus.INIT))
+      mock_context.add_expected_status_query_result(
+        self.create_mock_status_query_result(ScheduleStatus.RUNNING))
+      api = mock_context.get_api('west')
+      api.create_job.return_value = self.get_createjob_response()
+
+      with temporary_file() as fp:
+        fp.write(self.get_valid_config())
+        fp.flush()
+        cmd = AuroraCommandLine()
+        cmd.execute(['job', 'create', '--wait-until=RUNNING', 'west/bozo/test/hello',
+            fp.name])
+
+      self.assert_create_job_called(api)
+      self.assert_scheduler_called(api, mock_query, 2)
+      assert command_hook.ran_pre
+      assert command_hook.ran_post
+
+  def test_create_job_with_failed_hook(self):
+    GlobalCommandHookRegistry.reset()
+    command_hook = HookForTesting(False)
+    GlobalCommandHookRegistry.register_command_hook(command_hook)
+    mock_context = FakeAuroraCommandContext()
+    with patch('apache.aurora.client.cli.jobs.Job.create_context', return_value=mock_context):
+      mock_context.add_expected_status_query_result(
+        self.create_mock_status_query_result(ScheduleStatus.INIT))
+      mock_context.add_expected_status_query_result(
+        self.create_mock_status_query_result(ScheduleStatus.RUNNING))
+      api = mock_context.get_api('west')
+      api.create_job.return_value = self.get_createjob_response()
+
+      with temporary_file() as fp:
+        fp.write(self.get_valid_config())
+        fp.flush()
+        cmd = AuroraCommandLine()
+        result = cmd.execute(['job', 'create', '--wait-until=RUNNING', 'west/bozo/test/hello',
+            fp.name])
+
+        assert result == 1
+        assert api.create_job.call_count == 0
+        assert command_hook.ran_pre
+        assert not command_hook.ran_post
+
+  def test_load_dynamic_hooks(self):
+    GlobalCommandHookRegistry.reset()
+    hook_locals = GlobalCommandHookRegistry.load_project_hooks(
+        './src/test/python/apache/aurora/client/cli')
+    assert hook_locals['hooks'][0] in GlobalCommandHookRegistry.COMMAND_HOOKS
+
+  def test_successful_dynamic_hook(self):
+    GlobalCommandHookRegistry.reset()
+    hook_locals = GlobalCommandHookRegistry.load_project_hooks(
+        './src/test/python/apache/aurora/client/cli')
+    self.generic_test_successful_hook(hook_locals['hooks'][0])
+
+  def test_dynamic_hook_syntax_error(self):
+    with patch('logging.warn') as log_patch:
+      GlobalCommandHookRegistry.reset()
+      hook_locals = GlobalCommandHookRegistry.load_project_hooks(
+        './src/test/python/apache/aurora/client/cli/hook_test_data/bad_syntax')
+      log_patch.assert_called_with('Error compiling hooks file '
+          './src/test/python/apache/aurora/client/cli/hook_test_data/bad_syntax/AuroraHooks:
'
+          'invalid syntax (AuroraHooks, line 1)')
+
+  def test_dynamic_hook_exec_error(self):
+    with patch('logging.warn') as log_patch:
+      GlobalCommandHookRegistry.reset()
+      hook_locals = GlobalCommandHookRegistry.load_project_hooks(
+        './src/test/python/apache/aurora/client/cli/hook_test_data/exec_error')
+      log_patch.assert_called_with('Warning: error loading hooks file '
+          './src/test/python/apache/aurora/client/cli/hook_test_data/exec_error/AuroraHooks:
'
+          'integer division or modulo by zero')
+


Mime
View raw message