incubator-jspwiki-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harry Metske <harry.met...@gmail.com>
Subject Re: 2 last failing unit tests
Date Tue, 27 Oct 2009 17:16:53 GMT
I just did a fresh svn checkout, and ran "ant clean tests" from the cmdline.
This gives me 3 failures and 13 errors:
http://www.computerhok.nl/tmp/junit-noframes.html

The WikiEngineTest.testSpacedNames1() always fails (Linux versus
Mac/Windows)

Here's an overview of more tests :
http://www.computerhok.nl/tmp/jspwiki-testresult.html

regards,
Harry

2009/10/27 Andrew Jaquith <andrew.r.jaquith@gmail.com>

> Sounds like we have a few issues here:
>
> 1) Guitests. I'll see what I can find. Probably something minor. I
> know the "tests" target runs all test classes ending in "*Test" and
> ignores "AllTests", while Eclipse (and probably guitests) just runs
> the AllTests classes. It's likely that one or more of the AllTests
> classes is failing to include, oh, about 34 tests. :)
>
> 2) Graceful LDAP fail (inside the tests themselves). Any ideas on how
> to implement? The easy way would be to look for a localhost listener
> on 4890 (where the OpenLDAP test fixture listens) and then not run the
> tests if it isn't found. Should they FAIL or PASS in that case? It
> sounds like passing is the right thing to do.
>
> 3) Differences in your test pass rate versus mine. Not sure why your
> "ant tests" run would produce different results than mine. I did try
> running mine with a completely new, checked-out branch. Because I
> can't know what changes you might have in your local branch, could you
> check out a clean copy and diff the tree versus yours? SOMETHING is
> different. Also, I'd like to know what Harry and others are seeing.
> Gents, any clues?
>
> I agree that all three methods should return the same number of test
> cases, and pass/fail the same ways. I also agree that tests should be
> self-contained. That was part of the rationale for the Ant script
> tweaks I checked in recently.
>
> Eclipse, by the way, hasn't been reliable for me, for testing, for a
> while. I tend to exhaust memory somewhere around JSPWikiMarkupParser.
> But I haven't tried it in the last few months (i.e. before my massive
> bug-hunting campaign).
>
> Andrew
>
>
>
> On Tue, Oct 27, 2009 at 3:25 AM, Janne Jalkanen
> <Janne.Jalkanen@ecyrd.com> wrote:
> >> Interestingly, I applied your most recent checkins applied (and I have
> >> small one patch to JSPWikiMarkupParserTest that I haven't checked in).
> >> I am running 100% clean, with no errors. Total number of tests: 1024
> >> -- a nice round number. :)   WikiEngineTest.testOldVersionVars has
> >> been running fine for me for a while.
> >
> > There's no way it should've run, unless you have some old code/config
> files
> > lying around.  Can you check out a previous version to a clean directory
> and
> > see if it runs?
> >
> >> As a control case, I also checked out a new built from trunk, and
> >> simply typed 'ant tests'. I used a vanilla build with absolutely no
> >> customizations, even to build.properties. It ran completely clean also
> >> except for 1 JSPWikiMarkupParserTest test (because I haven't checked
> >> in that fix), 1024 tests total.
> >
> > Running the AllTests from Eclipse or with "ant guitests" results in 990
> test
> > cases.  "ant tests" is the only one giving 1024 tests, and I get 12
> failures
> > and 14 errors for it.  LdapAuthorizerTest, LdapUserDatabaseTest and
> > XMLUserDatabaseTest all fail with all tests.
> >
> > What I find odd is that guitests and tests targets should give the same
> > results, since they both are run from build.xml.
> >
> >> The only other item causing the discrepancy would be if you don't have
> >> a local LDAP server running for the LDAP tests. Those should cause, at
> >> most, 14 failures or errors. I'll add in some code to build.xml to set
> >> up the LDAP fixtures and/or disable the tests if the OpenLDAP
> >> executable isn't available.
> >
> > I think it's probably a better idea to do the test directly in the tests
> > itself.  The JCR TCK throws a NonExecutableException when the test case
> > cannot be executed (and this shows up as a passed test).
> >
> > I think it's important that all three methods give the same number of
> test
> > cases; if the number is not reliable, it's too easy to forget to run
> certain
> > tests.
> >
> > Also, I sometimes run all tests for a given package from within Eclipse.
> I'd
> > like the test cases to be self-contained enough so that I don't have to
> > remember which tests are supposed to run under which conditions.
> >
> > /Janne
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message