On Friday, August 3, 2012 at 11:51 AM, Jukka Zitting wrote:
Hi,On Fri, Aug 3, 2012 at 5:21 PM, Randall Hauch <email@example.com> wrote:The Jackrabbit TCK page  states that Jackrabbit's JCR Tests module isused within the official JSR-283 TCK  but is not the official TCK test.Obviously the former have been maintained and continue to improve, but theofficial JSR-283 TCK tests appear to have been released last on August 19,2009. Is it possible that the official JSR-283 TCK tests can be updated, andwould having a separate release schedule help with this in any way?The only way to update the official JSR-283 TCK is through the JCPmaintenance process .(If the official TCK tests cannot be updated, then I presume any project wantingto claim JSR-283 compliance would have to run the official TCKs and appeal eachof the older incorrect tests by stating the issue and perhaps using theupdated/corrected tests Jackrabbit JCR Tests module as "corrections".Yes, the appeal process is documented in  and relies on an excludelist of buggy tests maintained by the spec lead. In practice, if youhave pointers to relevant jackrabbit-jcr-tests bug reports and candemonstrate that your implementation passes the test after that issueis fixed, the spec lead will be happy to put the test case on theexclude list for you.
Is there anything that makes this easier with a non-updated TCK?It's a more light-weight process than doing a maintenance release through JCP.
How does Jackrabbit show spec and TCK compliance?We rely on the appeals process and exclude list as described above. Inpractice that means that we're good as long as Jackrabbit passes thelatest version of the tests in jackrabbit-jcr-tests.The official TCK and Jackrabbit's JCR tests unsurprisingly do not completelycover the specification, and doing so would require a significant amount ofeffort. However, there may be opportunity over time for projects to proposeadding new tests.Agreed. Especially with multiple active JCR implementations I thinkthere's a shared incentive to maintain compatibility beyond that ofwhat the official TCK tests. There's still plenty of place for healthycompetition on performance, scalability, maintainability and variousother features and "-ilities" that fall outside the scope of JCR.Thinking further, a standalone test codebase would also be a greatplace to cooperate on things like performance benchmarks or othertests that go beyond the scope of the JCR spec but that would stilladd value to implementors of content repositories.
If the JCR Tests' release cycle were independent from the Jackrabbit'srelease cycle, then the process of adding new tests and releasing updatedJCR tests might be easier. On the other hand, verification that the tests arevalid may become a bit harder.Not necessarily. If the tests were maintained outside the mainJackrabbit trunk, it would be easier set up a CI build that runs allupdates to the TCK codebase against the official JCR RI and lateststable versions of Jackrabbit, ModeShape and other JCR implementations(with excludes for tests that are known issues for each particularimplementation). That should make it much easier than now to catchproblems where the TCK codebase makes assumptions based on just asingle implementation.
For example, the ModeShape project has around 70 additional tests  thatwe'd be willing to donate. Most of these are around verifying administrationprivileges (e.g., registering namespaces, node types, etc., especially foranonymous users), versioning, and locking.That would be awesome!We also have some extra generic JCR tests, written for the jcr2spicomponent, that could and ideally should be used as a part of the TCK.
Additionally, the Oak effort is already running the JCR tests and hasrecently found/fixed issues, too. Can those participating in Oak offer howmuch they might expect to simply reuse vs. add new tests?
For now our goal is just to pass the already existing TCK tests, butI'm sure that over time we'll encounter cases where existing clientsthat assume only standard JCR functionality fail on Oak because ofsome problems in our code. Capturing such cases as TCK tests would bequite useful. http://www.day.com/content/day/en/products/jcr/jsr-283/_jcr_content/par/download_1/file.res/jsr-283-tck-appeal.pdfBR,Jukka Zitting