ace-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From build...@apache.org
Subject svn commit: r816324 - in /websites/staging/ace/trunk/content: ./ dev-doc/analysis/auditlog-analysis.html
Date Mon, 07 May 2012 14:34:34 GMT
Author: buildbot
Date: Mon May  7 14:34:33 2012
New Revision: 816324

Log:
Staging update by buildbot for ace

Added:
    websites/staging/ace/trunk/content/dev-doc/analysis/auditlog-analysis.html
Modified:
    websites/staging/ace/trunk/content/   (props changed)

Propchange: websites/staging/ace/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Mon May  7 14:34:33 2012
@@ -1 +1 @@
-1333079
+1335036

Added: websites/staging/ace/trunk/content/dev-doc/analysis/auditlog-analysis.html
==============================================================================
--- websites/staging/ace/trunk/content/dev-doc/analysis/auditlog-analysis.html (added)
+++ websites/staging/ace/trunk/content/dev-doc/analysis/auditlog-analysis.html Mon May  7
14:34:33 2012
@@ -0,0 +1,252 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
+<html lang="en">
+  <head>
+    <title>Audit Log Analysis</title>
+
+    <meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
+    <meta property="og:image" content="http://www.apache.org/images/asf_logo.gif" />
+
+    <link rel="stylesheet/less" type="text/css" href="/lib/bootstrap.less">
+    <link href="/css/prettify.css" rel="stylesheet">
+	<link href="/css/code.css" rel="stylesheet" media="screen">
+    <script src="/js/less-1.2.1.min.js" type="text/javascript"></script>
+    <script src="http://code.jquery.com/jquery-1.7.min.js"></script>
+    <script src="/js/prettify.js"></script>
+    
+    <script src="/js/bootstrap-alert.js"></script>
+    <script src="/js/bootstrap-dropdown.js"></script>
+    <script src="/js/bootstrap-tooltip.js"></script>
+    <script src="/js/bootstrap-alerts.js"></script>
+    <script src="/js/bootstrap-modal.js"></script>
+    <script src="/js/bootstrap-transition.js"></script>
+    <script src="/js/bootstrap-button.js"></script>
+    <script src="/js/bootstrap-popover.js"></script>
+    <script src="/js/bootstrap-twipsy.js"></script>
+    <script src="/js/bootstrap-buttons.js"></script>
+    <script src="/js/bootstrap-scrollspy.js"></script>
+    <script src="/js/bootstrap-typeahead.js"></script>
+    <script src="/js/bootstrap-carousel.js"></script>
+    <script src="/js/bootstrap-tab.js"></script>
+    <script src="/js/bootstrap-collapse.js"></script>
+    <script src="/js/bootstrap-tabs.js"></script>
+    
+    
+    
+    <script>
+    $(function () { prettyPrint() })
+    $().dropdown()
+    </script>
+  </head>
+
+  <body style="padding-top: 50px;">
+    <div class="navbar navbar-fixed-top">
+      <div class="navbar-inner">
+        <div class="container">
+          <a class="brand" href="/index.html">Apache ACE</a>
+          <ul class="nav">
+  <li class="dropdown">
+    <a href="#" class="dropdown-toggle" data-toggle="dropdown">News <b class="caret"></b></a>
+    <ul class="dropdown-menu">
+      <li>
+        <a href="/news.html">News</a>
+      </li>
+      <li>
+        <a href="/on-the-web.html">On the web</a>
+      </li>
+    </ul>
+  </li>
+  <li>
+    <a href="/downloads.html">Downloads</a>
+  </li>
+  <li class="dropdown">
+    <a href="#" class="dropdown-toggle" data-toggle="dropdown">User Documentation <b
class="caret"></b></a>
+    <ul class="dropdown-menu">
+      <li>
+        <a href="/user-doc/introduction.html">Introduction</a>
+      </li>
+      <li>
+        <a href="/user-doc/getting-started.html">Getting Started</a>
+      </li>
+      <li>
+        <a href="/user-doc/features.html">Features</a>
+      </li>
+      <li>
+        <a href="/user-doc/faq.html">FAQ</a>
+      </li>
+      <li>
+        <a href="/user-doc/support.html">Support</a>
+      </li>
+    </ul>
+  </li>
+  <li class="dropdown">
+    <a href="#" class="dropdown-toggle" data-toggle="dropdown">Developer Documentation
<b class="caret"></b></a>
+    <ul class="dropdown-menu">
+      <li>
+        <a href="/dev-doc/requirements/">Requirements</a>
+      </li>
+      <li>
+        <a href="/dev-doc/architecture.html">Architecture</a>
+      </li>
+      <li>
+        <a href="/dev-doc/analysis/">Analysis</a>
+      </li>
+      <li>
+        <a href="/dev-doc/design/">Design</a>
+      </li>
+      <li>
+        <a href="/dev-doc/coding-standards.html">Coding Standards</a>
+      </li>
+    </ul>
+  </li>
+  <li class="dropdown">
+    <a href="#" class="dropdown-toggle" data-toggle="dropdown">Get Involved <b class="caret"></b></a>
+    <ul class="dropdown-menu">
+      <li>
+        <a href="/get-involved/mailing-lists.html">Mailing Lists</a>
+      </li>
+      <li>
+        <a href="/get-involved/issue-tracking.html">Issue Tracking</a>
+      </li>
+      <li>
+        <a href="/get-involved/source-code.html">Source Code</a>
+      </li>
+      <li>
+        <a href="/get-involved/project-team.html">Project Team</a>
+      </li>
+    </ul>
+  </li>
+  <li class="dropdown">
+    <a href="#" class="dropdown-toggle" data-toggle="dropdown">Wiki <b class="caret"></b></a>
+    <ul class="dropdown-menu">
+      <li>
+        <a href="https://cwiki.apache.org/confluence/display/ACE/Board+Reports">Board
Reports <i class="icon-share-alt"></i></a>
+      </li>
+      <li>
+        <a href="https://cwiki.apache.org/confluence/display/ACE/Index">Homepage <i
class="icon-share-alt"></i></a>
+      </li>
+    </ul>
+  </li>
+  <li class="dropdown">
+    <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache <b class="caret"></b></a>
+    <ul class="dropdown-menu">
+      <li>
+        <a href="http://www.apache.org/licenses/">Licenses <i class="icon-share-alt"></i></a>
+      </li>
+      <li>
+        <a href="http://www.apache.org/security/">Security <i class="icon-share-alt"></i></a>
+      </li>
+      <li>
+        <a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship <i
class="icon-share-alt"></i></a>
+      </li>
+      <li>
+        <a href="http://www.apache.org/foundation/thanks.html">Thanks <i class="icon-share-alt"></i></a>
+      </li>
+    </ul>
+  </li>
+</ul>
+
+        </div>
+      </div>
+    </div>
+    <div class="container">
+      <p><a href="/"><i class='icon-home'></i> Home</a>&nbsp;&raquo&nbsp;<a
href="/dev-doc/">Dev-doc</a>&nbsp;&raquo&nbsp;<a href="/dev-doc/analysis/">Analysis</a></p>
+      <h1>Audit Log Analysis</h1>
+      <div class="clear"></div>
+      <div id="content"><p>An audit log is a full historic account of all events
that are relevant for a certain object. In this case, we keep audit logs of each target that
is managed by the provisioning server.</p>
+<h1 id="problem">Problem</h1>
+<p>The first issue is where to maintain the audit log. On the one hand, one can maintain
it on the target, but since the management agent talks to the server, it could keep the log
too.</p>
+<p>Then there is the question of how to maintain the log. What events should be in
it, and what is an event?</p>
+<p>Finally, the audit log should be readable and query-able, so people can review it.</p>
+<p>The following use cases can be defined:</p>
+<ul>
+<li>Store event. Stores a new event to the audit log.</li>
+<li>Get events. Queries (a subset of) events.</li>
+<li>Merge events. Merges a set of (new) events with the existing events.</li>
+</ul>
+<h1 id="context">Context</h1>
+<p>We basically have two contexts:</p>
+<ul>
+<li>Target, limited resources, so we should use something really "lean and mean".</li>
+<li>Server, scalable solution, expect people to query for (large numbers of) events.</li>
+</ul>
+<h1 id="possible-solutions">Possible solutions</h1>
+<p>As with all repositories, there should be one location where it is edited. In this
case, the logical place to do that is on the target itself, since that is where the changes
actually occur. In theory, the server also knows, but that theory breaks down if things fail
on the target or other parties start manipulating the life cycle of bundles. The target itself
can detect such activities.</p>
+<p>The next question is what needs to be logged. And how do we get access to these
events?</p>
+<p>When storing events, each event can get a unique sequence number. Sequence numbers
start with 1 and can be used to determine if you have the complete log.</p>
+<p>Assuming the target has limited storage, it might not be possible to keep the full
log available locally. There are a couple of reasons to replicate this log to a central server:</p>
+<ul>
+<li>space, as said the full log might not fit;</li>
+<li>safety, when the target is somehow (partly) erased or compromised, we don't want
to loose the log;</li>
+<li>remote diagnostics, we want to get an overview of the audit log without actually
connecting to the target directly.</li>
+</ul>
+<p>When replicating, the following scenarios can occur:</p>
+<ol>
+<li>The target has lost its whole log and really wants to (re)start from sequence number
1.</li>
+<li>The server has lost its whole log and receives a partial log.</li>
+</ol>
+<p>Starting with the second scenario, the server always simply collects incoming audit
logs, so its memory can be restored from any number of targets or relay servers that report
everything they know (again). Hopefully that will lead to a complete log again. If not, there's
not much we can do.</p>
+<p>The first scenario is potentially more problematic, since the target has no way
of knowing (for sure) at which sequence number it had arrived when everything was lost. In
theory it might ask (relay) servers, but even those might not have been up to date, so that
does not work. The only thing it can do here is: Start a new log at sequence number 1. That
means we can have more than one log in these cases, and that again means we need to be able
to identify which log (of each target) we're talking about. Therefore, when a new log is created,
it should contain some unique identifier for that log (an identifier that should not depend
on stored information, so for example we could use the current time in milliseconds, that
should be fairly unique, or just some random number).</p>
+<p>How to find the central server? Use the discovery service!? This is not that big
of a deal.</p>
+<p>Events should at least contain:</p>
+<ul>
+<li>a datestamp, indicating when the event occurred;</li>
+<li>a checksum and/or signature;</li>
+<li>a short, human readable message explaining the event;</li>
+<li>details:<ul>
+<li>in the form of a (possibly multi-line) document</li>
+<li>in the form of a set of properties</li>
+</ul>
+</li>
+</ul>
+<p>The server will add:</p>
+<ul>
+<li>the target ID of the target that logged the event.</li>
+</ul>
+<p>Storage will be resolve differently on the server and target. On the target, using
any kind of database would amount to having to include a considerable library, which makes
these solutions impractical there. We might want to consider something like that for the server
though. The options we have, are:</p>
+<ul>
+<li>Relational database</li>
+<li>Object database</li>
+<li>XML</li>
+<li>DIY</li>
+</ul>
+<p>How do events get logged?</p>
+<ul>
+<li>explicitly, our management agent calls an AuditLog service method;</li>
+<li>implicitly, by logging (certain) events in the system;</li>
+</ul>
+<p>Implicit algorithms can be build on top of the AuditLog service. What we need to
monitor is the life cycle layer, which basically means adding a BundleListener and an FrameworkListener.
Those capture all state changes of the framework. Technically we can either directly add those
listeners, or use EventAdmin if that is available.</p>
+<p>What would be the best way for the target to send audit log updates to the server?
I don't think we want the server to poll here, so the target should send updates (periodically).
So how does it know what to send?</p>
+<ul>
+<li>it could keep track of the last event it sent, sending newer ones after that;</li>
+<li>it could ask for the list of events the server has;</li>
+<li>it could send its highest log event number, and get back a list of missing events
on the server, and then respond with the missing events.</li>
+<li>it could just send everything.</li>
+</ul>
+<h1 id="discussion">Discussion</h1>
+<p>Having two layers for the audit log makes sense:</p>
+<ul>
+<li>The first, lowest, layer is the AuditLog service that gives access to the log.
On the one hand it allows people to log messages, on the other it should provide query access.
Those should be split into two different interfaces.</li>
+<li>The second layer can build on top of that. It can either be removed completely,
which means the responsibility for logging becomes that of the application (probably the management
agent). It can be implemented using listeners. Finally, it can be implemented using events.</li>
+</ul>
+<p>On the target we should implement a storage solution ourselves, to keep the actual
code small. The code should be able to log events quickly (as that will happen far more often
than retrieving them).</p>
+<p>Communication between the target and server should be initiated by the target. The
target can basically send two commands to the server:</p>
+<ol>
+<li>My audit log contains sequence number 4-8, tell me your numbers. The server then
responds (for example) with 1-6. This indicates we need to send 7-8.</li>
+<li>Here you have events 7-8, can you send me 1-3? The server stores its missing events,
and sends you the events it has (always check if what you get is what you requested).</li>
+</ol>
+<p>This is setup in this way so the same commands can also be used by relay servers
to replicate logs between server and target.</p>
+<h1 id="conclusion">Conclusion</h1>
+<ul>
+<li>The audit log is maintained on the target.</li>
+<li>On the target, we implement the storage mechanism ourselves to ensure we have a
solution with a very small footprint.</li>
+<li>On the server, we use an XStream based solution to store the logs of all the targets.</li>
+<li>Our communication protocol between target and (relay)server however, should probably
not rely on XML.</li>
+<li>Our communication protocol between server and (relay)server might rely on XML (determine
at design time what makes most sense).</li>
+</ul></div>
+      <hr>
+      <footer>
+        <p>Copyright &#169; 2012 <a href="http://www.apache.org/">The Apache
Software Foundation</a>, Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache
License, Version 2.0</a>.<br/>Apache ACE, the Apache ACE logo, Apache and the
Apache feather logo are trademarks of The Apache Software Foundation. All other marks mentioned
may be trademarks or registered trademarks of their respective owners.</p>
+      </footer>
+    </div>
+  </body>
+</html>



Mime
View raw message