lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shai Erera (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-3441) Add NRT support to facets
Date Wed, 21 Nov 2012 16:13:58 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502090#comment-13502090
] 

Shai Erera commented on LUCENE-3441:
------------------------------------

I added a TODO to consider switching to use DoubleBarrleLRUCache (to DirTaxoReader). Also
added some javadocs to DTR.doOpenIfChanged, noting about the cache sharing.

I ran all tests with 30 iterations on trunk and committed.
After I ported to 4x, I hit this failure:

org.apache.lucene.index.CorruptIndexException: Missing parent data for category 0
	at __randomizedtesting.SeedInfo.seed([A790471136DA8197:3D43CAE660B7F604]:0)
	at org.apache.lucene.facet.taxonomy.directory.ParentArray.initFromReader(ParentArray.java:116)
	at org.apache.lucene.facet.taxonomy.directory.ParentArray.<init>(ParentArray.java:66)
	at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyReader.<init>(DirectoryTaxonomyReader.java:92)
	at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyReader.doOpenIfChanged(DirectoryTaxonomyReader.java:238)
	at org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyReader.doOpenIfChanged(DirectoryTaxonomyReader.java:1)
	at org.apache.lucene.facet.taxonomy.TaxonomyReader.openIfChanged(TaxonomyReader.java:95)
	at org.apache.lucene.facet.taxonomy.directory.TestDirectoryTaxonomyReader.testOpenIfChangedManySegments(TestDirectoryTaxonomyReader.java:224)

NOTE: reproduce with: ant test  -Dtestcase=TestDirectoryTaxonomyReader -Dtests.method="testOpenIfChangedManySegments
{#66 seed=[A790471136DA8197:3D43CAE660B7F604]}" -Dtests.seed=A790471136DA8197 -Dtests.slow=true
-Dtests.locale=lt_LT -Dtests.timezone=Australia/South -Dtests.file.encoding=UTF-8
NOTE: test params are: codec=SimpleText, sim=DefaultSimilarity, locale=lt_LT, timezone=Australia/South
NOTE: Windows 7 6.1 build 7600 amd64/IBM Corporation 1.6.0 (64-bit)/cpus=2,threads=1,free=1071336,total=15596544
NOTE: All tests run in this JVM: [TestDirectoryTaxonomyReader]

Even though the test is single-threaded, it's not reproducible no matter if I use the master
seed or the iteration seed. I can reproduce quite easily though if I add -Dtests.iters=100
and -Dtests.failfast=yes -- but it makes it hard to debug ...

I'm digging. I won't revert the commit to trunk yet, hopefully I'll get to the bottom of it
quickly.
                
> Add NRT support to facets
> -------------------------
>
>                 Key: LUCENE-3441
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3441
>             Project: Lucene - Core
>          Issue Type: New Feature
>          Components: modules/facet
>            Reporter: Shai Erera
>            Assignee: Shai Erera
>            Priority: Minor
>         Attachments: LUCENE-3441.patch, LUCENE-3441.patch
>
>
> Currently LuceneTaxonomyReader does not support NRT - i.e., on changes to LuceneTaxonomyWriter,
you cannot have the reader updated, like IndexReader/Writer. In order to do that we need to
do the following:
> # Add ctor to LuceneTaxonomyReader to allow you to instantiate it with LuceneTaxonomyWriter.
> # Add API to LuceneTaxonomyWriter to expose its internal IndexReader
> # Change LTR.refresh() to return an LTR, rather than void. This is actually not strictly
related to that issue, but since we'll need to modify refresh() impl, I think it'll be good
to change its API as well. Since all of facet API is @lucene.experimental, no backwards issues
here (and the sooner we do it, the better).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message