incubator-sling-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Sling Website > Testing Sling-based applications
Date Tue, 08 May 2012 01:26:00 GMT
<html>
<head>
    <base href="https://cwiki.apache.org/confluence">
            <link rel="stylesheet" href="/confluence/s/2042/9/1/_/styles/combined.css?spaceKey=SLINGxSITE&amp;forWysiwyg=true"
type="text/css">
    </head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
    <h2><a href="https://cwiki.apache.org/confluence/display/SLINGxSITE/Testing+Sling-based+applications">Testing
Sling-based applications</a></h2>
    <h4>Page <b>edited</b> by             <a href="https://cwiki.apache.org/confluence/display/~bdelacretaz">Bertrand
Delacretaz</a>
    </h4>
        <br/>
                         <h4>Changes (9)</h4>
                                 
    
<div id="page-diffs">
                    <table class="diff" cellpadding="0" cellspacing="0">
    
            <tr><td class="diff-added-lines" style="background-color: #dfd;">h1.
Testing Sling-based applications <br> <br></td></tr>
            <tr><td class="diff-unchanged" >Automated testing of OSGi components
and services can be challenging, as many of them depend on other services that must be present
or simulated for testing. <br> <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >{toc} <br> <br></td></tr>
            <tr><td class="diff-changed-lines" ><span class="diff-changed-words">h<span
class="diff-deleted-chars"style="color:#999;background-color:#fdd;text-decoration:line-through;">1</span><span
class="diff-added-chars"style="background-color: #dfd;">2</span>.</span> Unit
tests <br></td></tr>
            <tr><td class="diff-unchanged" >When possible, unit tests are obviously
the fastest executing ones, and it&#39;s easy to keep them close to the code that they&#39;re
testing.  <br> <br>We have quite a lot of those in Sling, the older use the JUnit3
TestCase base class, and later ones use JUnit4 annotations. Mixing both approaches is possible,
there&#39;s no need to rewrite existing tests. <br> <br></td></tr>
            <tr><td class="diff-changed-lines" ><span class="diff-changed-words">h<span
class="diff-deleted-chars"style="color:#999;background-color:#fdd;text-decoration:line-through;">1</span><span
class="diff-added-chars"style="background-color: #dfd;">2</span>.</span> Tests
that use a JCR repository <br></td></tr>
            <tr><td class="diff-unchanged" >Utility classes from our [commons/testing|https://svn.apache.org/repos/asf/sling/trunk/bundles/commons/testing]
module make it easy to get a real JCR repository for testing. That&#39;s a bit slower
than pure unit tests, of course, but this only adds 1-2 seconds to the execution of a test
suite. <br> <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >Note that our utilities do not cleanup
the repository between tests, so you must be careful about test isolation, for example by
using unique paths for each test. <br> <br></td></tr>
            <tr><td class="diff-changed-lines" ><span class="diff-changed-words">h<span
class="diff-deleted-chars"style="color:#999;background-color:#fdd;text-decoration:line-through;">1</span><span
class="diff-added-chars"style="background-color: #dfd;">2</span>.</span> Mock
classes and services <br></td></tr>
            <tr><td class="diff-unchanged" >The next step is to use mock classes
services to simulate components that are needed for testing. This makes it possible to test
OSGi service classes without an OSGi framework. <br> <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >The problem with mocks is that it
can become hard to make sure you&#39;re actually testing something, and not just &quot;mocking
mocks&quot;. At a certain level of complexity, it becomes quicker and clearer to actually
start an OSGi framework for automated tests. <br> <br></td></tr>
            <tr><td class="diff-changed-lines" ><span class="diff-changed-words">h<span
class="diff-deleted-chars"style="color:#999;background-color:#fdd;text-decoration:line-through;">2</span><span
class="diff-added-chars"style="background-color: #dfd;">3</span>.</span> Injecting
services in private fields <br></td></tr>
            <tr><td class="diff-unchanged" >To inject (real or fake) services
in others for testing, without having to create getters and setters just for this, we use
a reflection-based trick, as in this example: <br> <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >{code} <br> <br></td></tr>
            <tr><td class="diff-changed-lines" ><span class="diff-changed-words">h<span
class="diff-deleted-chars"style="color:#999;background-color:#fdd;text-decoration:line-through;">1</span><span
class="diff-added-chars"style="background-color: #dfd;">2</span>.</span> Pax
Exam <br></td></tr>
            <tr><td class="diff-unchanged" >[Pax Exam|http://team.ops4j.org/wiki/display/paxexam/Pax+Exam]
allows you to easily start an OSGi framework during execution of a JUnit test suite. <br>
<br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >Such tests are obviously slower than
plain unit tests and tests that use mocks. Our installer integration tests, using Pax Exam,
take about a minute to execute on a 2010 macbook pro. <br> <br></td></tr>
            <tr><td class="diff-changed-lines" ><span class="diff-changed-words">h<span
class="diff-deleted-chars"style="color:#999;background-color:#fdd;text-decoration:line-through;">1</span><span
class="diff-added-chars"style="background-color: #dfd;">2</span>.</span> Sling
testing tools: server-side JUnit tests <br></td></tr>
            <tr><td class="diff-unchanged" >The [Sling testing tools|SLINGxSITE:Sling
Testing Tools] include a module that executes JUnit tests server-side, in a Sling instance.
This allows integration tests to run in a realistic environment, and could also be used for
self-testing production systems. <br> <br>As I write this, we are not using those
tools to test Sling itself, but we have a [complete example|https://svn.apache.org/repos/asf/sling/trunk/testing/samples/integration-tests]
of integration tests that use them to run server-side JUnit tests, including automatic setup
of the test Sling instance. <br> <br></td></tr>
            <tr><td class="diff-changed-lines" ><span class="diff-changed-words">h<span
class="diff-deleted-chars"style="color:#999;background-color:#fdd;text-decoration:line-through;">1</span><span
class="diff-added-chars"style="background-color: #dfd;">2</span>.</span> HTTP-based
integration tests <br></td></tr>
            <tr><td class="diff-unchanged" >The highest level of integration is
testing a complete Sling instance via its HTTP interface. <br> <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >One problem with these launchpad tests
is that the tests of all Sling modules are defined in a single testing module, the tests are
not co-located with the code that they test. This could be improved using the Sling testing
tools, by providing the tests in bundles that can be created from the same Maven modules that
the code that they test. <br> <br></td></tr>
            <tr><td class="diff-changed-lines" ><span class="diff-changed-words">h<span
class="diff-deleted-chars"style="color:#999;background-color:#fdd;text-decoration:line-through;">1</span><span
class="diff-added-chars"style="background-color: #dfd;">2</span>.</span> Summary
<br></td></tr>
            <tr><td class="diff-unchanged" >Combining the above testing techniques
has worked well for us in creating and testing Sling. Being able to test things at different
levels of integration has proved an efficient way to get good test coverage without having
to write too much boring test code. <br></td></tr>
    
            </table>
    </div>                            <h4>Full Content</h4>
                    <div class="notificationGreySide">
        <h1><a name="TestingSling-basedapplications-TestingSlingbasedapplications"></a>Testing
Sling-based applications</h1>

<p>Automated testing of OSGi components and services can be challenging, as many of
them depend on other services that must be present or simulated for testing.</p>

<p>This page describes the various approaches that we use to test Sling itself, and
introduces a number of tools that can help testing OSGi and HTTP-based applications.</p>

<div>
<ul>
    <li><a href='#TestingSling-basedapplications-TestingSlingbasedapplications'>Testing
Sling-based applications</a></li>
<ul>
    <li><a href='#TestingSling-basedapplications-Unittests'>Unit tests</a></li>
    <li><a href='#TestingSling-basedapplications-TeststhatuseaJCRrepository'>Tests
that use a JCR repository</a></li>
    <li><a href='#TestingSling-basedapplications-Mockclassesandservices'>Mock
classes and services</a></li>
<ul>
    <li><a href='#TestingSling-basedapplications-Injectingservicesinprivatefields'>Injecting
services in private fields</a></li>
</ul>
    <li><a href='#TestingSling-basedapplications-PaxExam'>Pax Exam</a></li>
    <li><a href='#TestingSling-basedapplications-Slingtestingtools%3AserversideJUnittests'>Sling
testing tools: server-side JUnit tests</a></li>
    <li><a href='#TestingSling-basedapplications-HTTPbasedintegrationtests'>HTTP-based
integration tests</a></li>
    <li><a href='#TestingSling-basedapplications-Summary'>Summary</a></li>
</ul>
</ul></div>

<h2><a name="TestingSling-basedapplications-Unittests"></a>Unit tests</h2>
<p>When possible, unit tests are obviously the fastest executing ones, and it's easy
to keep them close to the code that they're testing. </p>

<p>We have quite a lot of those in Sling, the older use the JUnit3 TestCase base class,
and later ones use JUnit4 annotations. Mixing both approaches is possible, there's no need
to rewrite existing tests.</p>

<h2><a name="TestingSling-basedapplications-TeststhatuseaJCRrepository"></a>Tests
that use a JCR repository</h2>
<p>Utility classes from our <a href="https://svn.apache.org/repos/asf/sling/trunk/bundles/commons/testing"
class="external-link" rel="nofollow">commons/testing</a> module make it easy to get
a real JCR repository for testing. That's a bit slower than pure unit tests, of course, but
this only adds 1-2 seconds to the execution of a test suite.</p>

<p>The <tt>RepositoryProviderTest</tt> in that module uses this technique
to get a JCR repository.</p>

<p>Note that our utilities do not cleanup the repository between tests, so you must
be careful about test isolation, for example by using unique paths for each test.</p>

<h2><a name="TestingSling-basedapplications-Mockclassesandservices"></a>Mock
classes and services</h2>
<p>The next step is to use mock classes services to simulate components that are needed
for testing. This makes it possible to test OSGi service classes without an OSGi framework.</p>

<p>We have a number of custom-written mock services in Sling, like <a href="https://svn.apache.org/repos/asf/sling/trunk/bundles/commons/testing/src/main/java/org/apache/sling/commons/testing/jcr/MockNodeType.java"
class="external-link" rel="nofollow">MockNodeType</a> for example. These handwritten
mocks implement just what's needed for their tests, so they might not be reusable as is.</p>

<p>In other cases we use <a href="http://www.jmock.org/" class="external-link" rel="nofollow">jmock</a>
to help create mock objects without having to write much code - such mocking libraries take
care of the plumbing and allow you to write just the bits of code that matter (often with
funny syntaxes). The tests of the <a href="https://svn.apache.org/repos/asf/sling/trunk/bundles/extensions/event/"
class="external-link" rel="nofollow">org.apache.sling.event</a> bundle, for example,
make extensive use of such mock services.</p>

<p>The problem with mocks is that it can become hard to make sure you're actually testing
something, and not just "mocking mocks". At a certain level of complexity, it becomes quicker
and clearer to actually start an OSGi framework for automated tests.</p>

<h3><a name="TestingSling-basedapplications-Injectingservicesinprivatefields"></a>Injecting
services in private fields</h3>
<p>To inject (real or fake) services in others for testing, without having to create
getters and setters just for this, we use a reflection-based trick, as in this example:</p>

<div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader"
style="border-bottom-width: 1px;"><b>setting a private field via reflection</b></div><div
class="codeContent panelContent">
<pre class="code-java">
<span class="code-comment">// set resource resolver factory
</span><span class="code-comment">// in a ServletResolver object which has a <span
class="code-keyword">private</span> resourceResolverFactory field
</span>
ServletResolver servletResolver = ....
<span class="code-object">Class</span>&lt;?&gt; resolverClass = servletResolver.getClass().getSuperclass();
<span class="code-keyword">final</span> java.lang.reflect.Field resolverField
= resolverClass.getDeclaredField(<span class="code-quote">"resourceResolverFactory"</span>);
resolverField.setAccessible(<span class="code-keyword">true</span>);
resolverField.set(servletResolver, factory);
</pre>
</div></div>

<h2><a name="TestingSling-basedapplications-PaxExam"></a>Pax Exam</h2>
<p><a href="http://team.ops4j.org/wiki/display/paxexam/Pax+Exam" class="external-link"
rel="nofollow">Pax Exam</a> allows you to easily start an OSGi framework during execution
of a JUnit test suite.</p>

<p>We currently use it for our <a href="https://svn.apache.org/repos/asf/sling/trunk/installer/it"
class="external-link" rel="nofollow">Sling installer integration tests</a> for example.
As parts of the installer interact directly with the OSGi framework, it felt safer to test
it in a realistic situation rather than mock everything.</p>

<p>Such tests are obviously slower than plain unit tests and tests that use mocks. Our
installer integration tests, using Pax Exam, take about a minute to execute on a 2010 macbook
pro.</p>

<h2><a name="TestingSling-basedapplications-Slingtestingtools%3AserversideJUnittests"></a>Sling
testing tools: server-side JUnit tests</h2>
<p>The <a href="/confluence/display/SLINGxSITE/Sling+Testing+Tools" title="Sling
Testing Tools">Sling testing tools</a> include a module that executes JUnit tests
server-side, in a Sling instance. This allows integration tests to run in a realistic environment,
and could also be used for self-testing production systems.</p>

<p>As I write this, we are not using those tools to test Sling itself, but we have a
<a href="https://svn.apache.org/repos/asf/sling/trunk/testing/samples/integration-tests"
class="external-link" rel="nofollow">complete example</a> of integration tests that
use them to run server-side JUnit tests, including automatic setup of the test Sling instance.</p>

<h2><a name="TestingSling-basedapplications-HTTPbasedintegrationtests"></a>HTTP-based
integration tests</h2>
<p>The highest level of integration is testing a complete Sling instance via its HTTP
interface.</p>

<p>We use this technique to test Sling itself: the <a href="https://svn.apache.org/repos/asf/sling/trunk/launchpad/integration-tests"
class="external-link" rel="nofollow">launchpad/integration-tests</a> module defines
the tests (462 of them as I write this), and the <a href="https://svn.apache.org/repos/asf/sling/trunk/launchpad/testing"
class="external-link" rel="nofollow">launchpad/testing</a> module executes them,
after setting up a Sling instance from scratch (which is quite easy as Sling is just a runnable
jar). </p>

<p>A simple mechanism (described in README files in these modules) allows individual
tests to be executed quickly against a previously started Sling instance, to be able to write
and debug tests efficiently.</p>

<p>The test code could be made simpler using the fluent HTTP interfaces defined in the
Sling testing tools described above, but the launchpad tests were written before that module
was created, and as they're stable there's no reason to rewrite them. If you're planning on
using this technique for your own applications, we recommend looking at the Sling testing
tools instead of these "legacy" tests - but the basic technique is the same.</p>

<p>One problem with these launchpad tests is that the tests of all Sling modules are
defined in a single testing module, the tests are not co-located with the code that they test.
This could be improved using the Sling testing tools, by providing the tests in bundles that
can be created from the same Maven modules that the code that they test.</p>

<h2><a name="TestingSling-basedapplications-Summary"></a>Summary</h2>
<p>Combining the above testing techniques has worked well for us in creating and testing
Sling. Being able to test things at different levels of integration has proved an efficient
way to get good test coverage without having to write too much boring test code.</p>
    </div>
        <div id="commentsSection" class="wiki-content pageSection">
        <div style="float: right;">
            <a href="https://cwiki.apache.org/confluence/users/viewnotifications.action"
class="grey">Change Notification Preferences</a>
        </div>
        <a href="https://cwiki.apache.org/confluence/display/SLINGxSITE/Testing+Sling-based+applications">View
Online</a>
        |
        <a href="https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=27846122&revisedVersion=7&originalVersion=6">View
Changes</a>
                |
        <a href="https://cwiki.apache.org/confluence/display/SLINGxSITE/Testing+Sling-based+applications?showComments=true&amp;showCommentArea=true#addcomment">Add
Comment</a>
            </div>
</div>
</div>
</div>
</div>
</body>
</html>

Mime
View raw message