jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randall Hauch <rha...@gmail.com>
Subject JSR-283 official TCK and the 'jackrabbit-jcr-tests'
Date Thu, 07 Jun 2012 14:00:11 GMT
First of all, I'd like to thank the whole Jackrabbit community for maintaining and adding to
the "jackrabbit-jcr-tests" suite of unit tests. IIUC, these tests started out as the suite
that is used by the JSR-283 TCK, but which now include quite a few fixes and enhancements.
Having these tests is a huge plus for Jackrabbit but also for other JCR implementations (including
ModeShape [1][2] and Jackrabbit's own Oak project [3]), since this large suite of independent
tests helps any implementation to provide behavior consistent with the specification.

The official JSR-283 TCK tests [4] was released in August 2009, and IIUC was based on one
of the "jackrabbit-jcr-tests" module in one of the 2.0.x releases of Jackrabbit. Since then,
the official TCK hasn't been updated or corrected, while the "jackrabbit-jcr-tests" have undergone
quite a few corrections and changes. (I recently learned of this subtle distinction.)

Specifically, since the 2.0 release of Jackrabbit, all of the following Jackrabbit issues
changed some part of the unit tests in "jackrabbit-jcr-tests" [5]:

JCR-3316: fix namespace name to be an absolute URI
JCR-3313 Changed a ColumnTest test to only check for required columns
JCR-3307: check supported option OPTION_ACTIVITIES_SUPPORTED
JCR-3301: fix typos in test method names and associated config files
JCR-3302: add missing refresh() call on test session
JCR-3300: make RSessionAccessControlTest extend the proper base class so that the repo option
check happens
JCR-3207: add TCK test for Info map of NODE_MOVED event on node reordering
JCR-3205 - Missing support for lock timeout and ownerHint in jcr-server
JCR-3196: reduce deprecation warning noise in TCK testsReduce warning noise
JCR-3195: fix assumptions about null-ness of lock token
JCR-3177: Remove jdk 1.4 restriction for jcr-tests
JCR-3051: AbstractLockTest.testLockExpiration fails intermittently
JCR-2903: Session.importXml should close the input stream (as to JSR 283/JCR 2.0)
JCR-2756: Shareable node test failures
JCR-2719: Incorrect outer join TCK tests
JCR-2693: Logging per test case
JCR-2665: JCR Test for Adding Node Type Tests That Abstract Nodes Can Be Added as Children,
contrary to JCR 2.0 specification
JCR-2663: JCR unit tests use invalid queries
JCR-2662: check for repository support of journaled observation, otherwise skip test
JCR-2648: PropertyImpl.getNode() and NamePropertyTest use different exception than documented
in the JCR API JavaDoc
JCR-2536: spi2davex: InvalidItemStateException not properly extracted from ambiguous response
JCR-2565: spi2dav: Overwrite header T specified for MOVE and COPY causes
JCR-2515: ISO8601 uses default DecimalFormat constructor using locale specific digits
JCR-2490: jackrabbit wrongly think nodetype is changed on nodetype re-registration
JCR-2104: JSR 283 Full Versioning - checkpoint must be aware of current activity for the checkout
JCR-515: Enhance test data

IIUC most/all of them are still unfixed in the official TCK, and this can cause quite a bit
of grief for other implementations. Plain and simple, the "jackrabbit-jcr-tests" module contains
tests that are more accurately verifying the expectations of the JSR-283 specification than
the official TCK. And this is why ModeShape adopted has for years used the "jackrabbit-jcr-tests"
in its builds, but in doing so we've run into some challenges.

Since the "jackrabbit-jcr-tests" module is part of the Jackrabbit codebase, it is released
along with the rest of Jackrabbit. Of course, this makes perfect sense for the Jackrabbit
community. But considering that the "jackrabbit-jcr-tests" are used by at least two other
OSS projects as essentially a "more correct and maintained" proxy for the TCK tests, this
inclusion in the Jackrabbit releases means the TCK tests are not as frequently released as
we'd hope.

Is there any interest in separating the "jackrabbit-jcr-tests"? I'm not sure if that means
a completely separate (sub)project, or whether there are better alternatives (e.g., a simple
project on GitHub?). Separating the tests would have a number of benefits:
The "TCK unit tests" could be released on their own schedule, with version numbers that are
semantically related to the changes made within that release
The JIRA issues would better reflect the work that's been done (e.g., release notes), the
number of changes, and the outstanding issues (see below)
Emphasis can be placed on encouraging reuse and participation by other projects, and ensuring
that the tests are as correct as possible
This can possibly form a basis for either updating the official TCK (if that's even possible)
or providing a form that excludes/disables the incorrect tests and includes fixed tests (per
the TCK appeals process), making the process much easier for those submitting results (with
appeals) as well as those reviewing/approving results.
Perhaps allow for more tests to be added, though this may be difficult if these unit tests
do form the basis for an updated official TCK. (ModeShape has a number of tests that we haven't
yet submitted, since we thought until recently that the official TCK was periodically updated
with the latest "jackrabbit-jcr-tests".)  

As mentioned, there are currently 10 others issues that are still open, in progress, or have
a patch available [6]:

JCR-3321 Strange XPath query in OrderByMultiTypeTest.testMultipleOrder
JCR-3302 Remove incorrect assumptions with respect to visibility of changes done by other
JCR-3301 Remove incorrect test assumptions
JCR-3196 reduce deprecation warning noise in TCK tests
JCR-3096 AbstractObservationTest does not cover JCR 2.0 functionality
JCR-2756 Shareable node test failures
JCR-2661 Two JCR unit tests expect delete events for nodes under deleted node, contrary to
JCR 2.0 specification
JCR-2348 Exclude problematic node types from tests
JCR-3322 add TCK coverage of isNodeType(expandedName)
JCR-2666 JCR TCK Test for Restoring Version Tests That Versionable Child Is also Restored,
contrary to JCR 2.0 specification

(JSR-1991 is also still open and in the report, but has to do more with the OSGi bundling
of the JAR.)

I have no doubt that at a minimum the Jackrabbit, ModeShape, and Oak communities would very
much like to get these resolved and would continue to work together to build upon, improve
and maintain the JCR unit tests. But I believe that this will be even better if the JCR unit
tests can be separated.

Thanks for considering, and I look forward to discussing the possibilities.

Best regards,

Randall Hauch
Project lead, ModeShape

[1] http://modeshape.org
[2] https://docs.jboss.org/author/display/MODE/Home
[3] http://wiki.apache.org/jackrabbit/Jackrabbit%203
[4] http://www.day.com/day/en/products/jcr/jsr-283.html
[5] https://github.com/apache/jackrabbit/commits/trunk/jackrabbit-jcr-tests/src
[6] https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+JCR+AND+component+%3D+jackrabbit-jcr-tests+AND+status+in+%28Open%2C+%22In+Progress%22%2C+%22Patch+Available%22%29+ORDER+BY+status+ASC%2C+key+DESC

View raw message