lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-442) TestIndexModifier.testIndexWithThreads is not valid?
Date Wed, 26 Oct 2005 01:57:56 GMT
    [ http://issues.apache.org/jira/browse/LUCENE-442?page=comments#action_12355911 ] 

Hoss Man commented on LUCENE-442:
---------------------------------

Daniel: I used the following script to run through 50 iterations of the test...

#!/bin/sh
for ((i=0; i < 50; i++)); do {
  ant test -Dtestcase=TestIndexModifier;
  mv build/test/TEST-org.apache.lucene.index.TestIndexModifier.xml build/test/TEST-org.apache.lucene.index.TestIndexModifier.xml.$i;
};
done
grep -l "failure message" build/test/TEST-org.apache.lucene.index.TestIndexModifier.xml.*

...prior to upgrading to the changes you made, 30 of the 50 test runs failed (with the values
being off by one or two in each case).  After upgrading to TestIndexModifier.java v328151,
0 of the 50 runs failed.  

It's not a proof of correctness, but it certainly bodes well.

I also want to clarify the scenerio I described earlier: I'm an idiot.  I completely missed
that each call to testIndexInternal(int) blocks waiting for the threads to finish -- for some
reason I thought all three calls were happening concurrently.  So the types of scenerios I
was initially thinking could cause problems were completely non-existent -- and it looks like
you fixed all the ones Yonik noticed.

(I do think Yonik has a good point however regarding a test with no synchronization at all
-- this should be possible if each instant of IndexThread maintaining it's own id and idStack
variables, using  'threadNumber + "-" id++' to generate new doc identifiers, and only ever
deleted documents it added).

> TestIndexModifier.testIndexWithThreads is not valid?
> ----------------------------------------------------
>
>          Key: LUCENE-442
>          URL: http://issues.apache.org/jira/browse/LUCENE-442
>      Project: Lucene - Java
>         Type: Bug
>   Components: Search
>     Versions: 1.9
>     Reporter: Hoss Man

>
> I recently started playing with the trunk of SVN, and noticed that intermitently, TestIndexModifier.testIndexWithThreads
(revision 292010) would fail.
> The basic premise of the test seems to be that 3 pairs of IndexThread instances can be
started in parallel, each pair using the same instance of IndexModifier to concurrently and
randomly add/delete/optimize a single FSDirectory index.  
> The test is considered a success if the sum of additions-deletions recorded by each pair
of threads equals the final docCount() for the IndexModifier instance used by that pair of
threads.
> Now I freely admit that I'm not 100% familiar with the code for IndexModifier, but at
a glance, the basic premise seems to be: 
>    a) If method for IndexWriter is called, open it if needed, close the IndexReader first
if needed.
>    b) if method for IndexReader is called, open it if needed, close the IndexWriter first
if needed.
> If I'm understnading that correctly, I see no reason to assume this test will pass. 

> It seems like there could be plenty of scenerios in which the number of additions-deletions
!= docCount(). The most trivial example I can think of is:
>    1) the first IndexThread instance which has a chance to run adds a document, and optimizes
before any other IndexThreads ever open the Directory.
>    2) a subsequent pair of IndexThread instances open their IndexModifier instance before
any documents are deleted.
>    3) the IndexThread instances from #2 do nothing but add documents
> ...that pair of IndexThreads is now garunteed to have recorded a differnet number of
additions then the docCount returned by their IndexModifier.
> Am I missing something, or should this test be removed?

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message