db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Myrna van Lunteren <m.v.lunte...@gmail.com>
Subject Re: Adding new documentation for debugging test failures in the test framework
Date Thu, 28 Apr 2005 08:49:18 GMT
Hi,


Kathey wrote: 
> Shreyas would you mind generating the html from  forrest and sending
> that? It is hard to review the xml without the  forrest build setup and
> haven't had a chance to get that set up yet.

I finally got around to getting to look at this myself...and I
couldn't deal with reading this in xml (my browser just displayed the
source). So, I loaded the thang in my old forrest 5 install & I'm
attaching the html for Kathey. :-)
Of course none of the tab-stuff works, but it makes for easier reading
of the doc.

And here are my 2 cents as to the contents:

It appears that mostly Kathey's email was just put into the xml
unchanged? Some formatting changes would definitely help readability.

Anyways, it's great to see this doc. 

Here's some specific changes I would suggest:

- typos:
  - Note - framewrok instead of framework
  -  as Jack said, on 1 line of steps to be followed - Frist instead
of First. Fro instead of for
  -  line 3: netwrok server instead of network server

- other: 
   - I suggest a carriage return after each first sentence for each
step. I think it's easier readable. Just a suggestion, though.
   -  I would suggest not to have mytest in quotes - or mention that
the quotes are just to indicate it's not a real test. ok, this is
probably obvious, but it struck me as being incorrect syntax when I
first read the doc.
   - change the line for network server to include derbyclient like so:
       "For the netwrok server you would need to add a framework
property -Dframework=DerbyNetClient (or -Dframework=DerbyNet if you're
testing the IBM Universal Driver - "jcc")
   - section 4: split out the a, b, c so they're below each other.
again just for readability
   - section 4 contains the suggestion to keep a clean client for comparison.
     I think this would be better to put right after 'The old
behavior'. (a). It's a really valuable step, to go back to what
happens without your change. Also, I don't think 'client' is
appropriate with svn, I suggest: 'setup' instead.
  - when it seems the old behavior was incorrect, it may be valuable
to see why that piece of code was put in & what related changes went
into the test. To do this, use svn log <filename>. To see all changes
to (other) files that went in for a particular revision, use: svn log
-v -r <revisionnumber>.
   (Of course, it could be erroneous accepted behavior stems from
before contribution to Apache. In that case, svn won't help & you'll
have to post to the list - possibly an IBM employee can backtrack, or
folks on the list can confirm the behavior is not ok.)
  - section 5: again, split up the a, b, c
  - section 6: ditto.
  - as Jack said, derby.log is also a good source for analyzing
trouble. If you've found the trouble code in the test, and made a
small subset (e.g. a tiny .sql that you can run in ij, or a small java
program that can be debugged easily) it will help to add at least the
following properties to a (to be newly created) derby.properties file:
      derby.infolog.append=true
      derby.language.logStatementText=true
      derby.stream.error.logSeverityLevel=0
   I suggest putting something about this in section 4. 
  - I suggest adding a bit about debugging the tests in an IDE - for
example eclipse, something like this:
"If you can't easily reproduce or analyze the problem within a small
ij case or program, you could try debugging the tests in - for example
- eclipse. To do so, you need to add the following to the vm arguments
for org.apache.derbyTesting.functionTests.harness.RunTest (your main
class):
     -Dclasspath=<mytrunk>/classes;<mytrunk>/tools/java/jakarta-oro-2.0.8.jar
 -Duser.dir=c:/testdir -Duseprocess=false
   Without useprocess=false, the test will kick off a new jvm process
to actually run the test and you cannot debug those threads.
  But debugging a small test case is much better"
- step 6: somewhere in here should go that you should report the
environment you ran in, especially. jvm version and os.
 

Thx for making this doc!

Myrna

Mime
View raw message