river-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Jones (JIRA)" <j...@apache.org>
Subject [jira] Commented: (RIVER-304) Reactivate River jtreg tests
Date Fri, 01 May 2009 21:31:31 GMT

    [ https://issues.apache.org/jira/browse/RIVER-304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705123#action_12705123
] 

Peter Jones commented on RIVER-304:
-----------------------------------

I'm not sure of the exact cause of the permission failures Peter
Firmstone is seeing with the River jtreg tests, but for the record,
many of these tests have security policy files that expect that the
following JAR files from the River build:

	lib/jsk-platform.jar
	lib/jsk-resources.jar
	lib/jsk-lib.jar
	lib/phoenix-init.jar
	lib-ext/jsk-policy.jar

are in the extensions directory ([jre/]lib/ext) of the JRE used to
execute the tests.  Also note that when jtreg sets a security policy
for test execution-- such as due to the /policy action option-- it does
so with the behavior of prefixing the java.security.policy system
property with "=", meaning that the JRE's system and user security
policies are ignored.  See the doc for the /policy option here:
    http://openjdk.java.net/jtreg/tag-spec.txt
And the doc for the java.security.policy system property here:
    http://java.sun.com/javase/6/docs/technotes/guides/security/PolicyFiles.html#DefaultLocs
The security policy files for the River jtreg tests include a grant of
AllPermission to the JRE's extentions directory, similar to the defualt
JRE security policy, so that jsk-platform.jar, etc., will have the
permissions they need.

For comparison, I just ran the River jtreg suite myself against a fresh
build of the River trunk, using JDK 6u11, with the following results:

Test results: passed: 130; failed: 8

Here are some comments about the 8 tests that failed for me:

FAILED: net/jini/config/ConfigurationFile/TestParser/TestParser.java   

This is the only failure that I don't understand at this point.  (This
test is a script-driven suite of unit tests of the ConfigurationFile
parser.)

FAILED: net/jini/jeri/http/echo/EchoImpl.java

This test failed for me because it couldn't resolve the host name
"jiniproxy", where-- for one variation of the test-- it is expecting to
find an HTTP proxy, in order to test HTTP proxy support for the
net.jini.jeri.http transport implementation.  Clearly, this was an
assumption about the internal Sun environment which doesn't even hold
here/there anymore.

FAILED: net/jini/jeri/kerberos/RegressionTests/runListenEndpointTest.sh

Another failed assumption of the previous internal Sun environment:
this test is expecting to find a Kerberos KDC at the host name
"jiniautot.east.sun.com".

FAILED: net/jini/jeri/kerberos/UnitTests/runTestEndpoints.sh
FAILED: net/jini/jeri/kerberos/UnitTests/runTestPerformance.sh

These tests failed because they attempt to invoke the JRE's "kinit"
tool using the internal sun.security.krb5.internal.tools.Kinit API,
from which it was available in 1.4.x and 5.0 JRE versions.  The Java
version of this tool was removed from the Solaris and Linux JREs for
version 6 because there is a native kinit on those platforms, although
it remains in the Windows JRE.

Beyond this kinit problem, though, these Kerberos tests would surely
have failed expecting to find a KDC at "jiniautot.east.sun.com" like
the previous two.

FAILED: net/jini/jeri/tcp/localHostExposure/LocalHostExposure.java
FAILED: net/jini/jeri/transport/multihomed/Multihomed.java
FAILED: net/jini/jeri/transport/multihomed/runMultihome.sh

These tests failed during the compilation phase because of this issue
discussed recently here:

http://mail-archives.apache.org/mod_mbox/incubator-river-dev/200904.mbox/%3C20090421151237.GA19950@east%3E

That is, the dependency of certain tests (in the QA suite and the jtreg
suite) on the JDK-internal sun.net.spi.nameservice API, which changed
incompatibly between JDK 5.0 and JDK 6.  The dependencies in the QA
suite were modified by Brian Murphy to work with the JDK 6 version, but
not the dependencies in the jtreg suite.  It's straightforward to
modify them to work with one version of the JDK or the other, but
somewhat tricky to make them work with both at the same time.  For
consistency, it probably makes sense to make them work with JDK 6 at
least.


> Reactivate River jtreg tests
> ----------------------------
>
>                 Key: RIVER-304
>                 URL: https://issues.apache.org/jira/browse/RIVER-304
>             Project: River
>          Issue Type: Test
>         Environment: JDK 1.5 or later with jtreg test suite http://www.openjdk.org/jtreg/

>            Reporter: Peter Firmstone
>         Attachments: ant.html, jtreg_stdout_errout.txt, JTreport.tgz, JTwork.tgz
>
>
> From a recent discussion on river-dev:
> Peter Firmstone wrote:
> > Using the GPLv2 version of jtreg is ok as a platform requirement for the tests,
we just can't distribute it with River.
> >
> > Peter Jones wrote:
> >> On Tue, Apr 21, 2009 at 06:27:18PM +0200, Jonathan Costers wrote:
> >>  
> >>> Something a bit off-topic: the "jtreg tests" are mentioned in the
> >>> discussion you linked to. How do these differ from the other harness/QA
> >>> tests? I must say I haven't really looked at them deeply, but I did
> >>> notice them and that they are separate from the QA suite ...
> >>> For the moment the source just sits there .. Nothing is even compiled.
> >>> Would you be able to give some pointers?
> >>>     
> >>
> >> Sure.  They are written to be run with "jtreg", the test harness used
> >> for regression & unit tests for Sun's JDK.  These days there is a
> >> version of jtreg available under GPLv2 as part of the OpenJDK project,
> >> here:
> >>
> >>     http://www.openjdk.org/jtreg/
> >>
> >> The use of this test framework in addition to the primary Jini QA
> >> framework is historical: some of the APIs added to version 2.0 of the
> >> Jini starter kit-- such as JERI and the related security model,
> >> preferred classes, the configuration stuff-- were originally developed
> >> for the JDK, mostly under JSRs 76 & 78, and thus their implementations
> >> initially had tests written for the jtreg framework.  When those APIs
> >> and implementations were moved to the Jini starter kit, those jtreg
> >> tests came with them, and some new tests in those areas continued to
> >> be added to this jtreg suite.
> >>
> >> The essential jtreg model is very simple: a test is a tagged class
> >> (source file) with a normal "main" method-- if that method completes
> >> normally, the test passes; if it throws an exception (or times out, or
> >> the JVM crashes...), the test fails.  The jtreg goal was to set a very
> >> low barrier to move standalone test cases or example code into the
> >> framework.  The framework does specify more options and nuances, but
> >> it's still pretty simple overall:
> >>
> >>     http://www.openjdk.org/jtreg/tag-spec.txt
> >>
> >> which is quite nice for some things-- of course it doesn't have
> >> anything like the power of the Jini QA framework for testing of
> >> distributed services, etc.  And the Jini jtreg suite has accreted an
> >> unfortunately somewhat ad hoc infrastructure library of its own, in
> >> the "qa/jtreg/testlibrary" directory.  Also, I think that it still has
> >> a few assumptions about being run within Sun's internal network, like
> >> that certain services (a KDC?) are provided by certain host names.
> >>
> >> You just see source files because the harness is responsible for
> >> building them at test execution time.  The jtreg implementation is
> >> built as a layer on top of the JavaTest framework (a much more complex
> >> test framework used for the JCK among other things), which has the
> >> same build-at-test-execution-time model.  (This is nice in that
> >> breaking the compilation of one test doesn't prevent executing other,
> >> unaffected tests-- each test is isolated all the way to its source.)
> >>
> >> I'm not sure how the GPLv2 status of the jtreg implementation
> >> available through the OpenJDK project affects the ability to use it to
> >> run these tests for River.  A nice aspect of the jtreg model's
> >> simplicity is that the test classes themselves do not need to link to
> >> or otherwise refer to any test framework APIs-- there are no such
> >> APIs.  (In theory each of these tests can be run as is, with the right
> >> class paths and javac & java commands.)  At one time the engineering
> >> lead for the Jini QA harness had prototyped adding support for
> >> jtreg-style tests to the Jini harness, and I think that he had gotten
> >> it more or less working, but that effort was dropped for reasons I
> >> can't recall-- although I would guess that it didn't seem like a
> >> priority at the time because jtreg itself was available internally.
> >>
> >> -- Peter
> >>
> >>   

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message