lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Issues with Grouping
Date Sat, 09 Jul 2011 11:16:05 GMT
If possible (but it may be hard here since you don't have direct
acces...) it's better to purge only the readers you know are expected
to have caused insanity?

Otherwise you could mask a real problem.

FieldCache insanity is bad, so, when it's caused we need to make sure
we understand the root cause, which we do in this case since faceting
is knowingly able to pull FC against the top-level reader while
Grouping will alsways pull FC per segment.

Mike McCandless

http://blog.mikemccandless.com

On Sat, Jul 9, 2011 at 4:52 AM, Bill Bell <billnbell@gmail.com> wrote:
> Yonik and Mike,
>
> It seems to happen when you facet and group.
>
> What do you suggest I do? Should the code change to switch to one or the
> other mode when
> we are using facet=true and group=true?
>
> For the test to work without erroring, shall I just added:
>
>    } finally {
>       FieldCache.DEFAULT.purgeAllCaches();
>    }
>
> I am not sure if I am masking anything, since Yonik said it is normal.
>
>
>
>
> On 7/5/11 4:30 AM, "Michael McCandless" <lucene@mikemccandless.com> wrote:
>
>>You can purge entries from the field cache before your test finishes?
>>
>>Ie call FieldCache.DEFAULT.purge(IR).
>>
>>The problem is that will mask any real problems where insanity is
>>unexpectedly being created.
>>
>>Or, maybe switch test over to using per-segment faceting?  Then (I
>>think?) no insanity should be created.  Insanity is bad since it means
>>double (or more) RAM is being used...
>>
>>Mike McCandless
>>
>>http://blog.mikemccandless.com
>>
>>On Tue, Jul 5, 2011 at 3:16 AM, Bill Bell <billnbell@gmail.com> wrote:
>>> OK?
>>>
>>> I get an error when I get the FC Insanity issue in SOLR-2242...
>>>
>>> ant -Dtestcase=NumFacetTermsFacetsTest test
>>>
>>> If it is normal, how do I turn off the FAILURE in the test case?
>>>
>>> [junit] Testsuite: org.apache.solr.request.NumFacetTermsFacetsTest
>>>    [junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 4.072 sec
>>>    [junit]
>>>    [junit] ------------- Standard Error -----------------
>>>    [junit] NOTE: reproduce with: ant test
>>> -Dtestcase=NumFacetTermsFacetsTest
>>> -Dtestmethod=testNumFacetTermsFacetCounts
>>> -Dtests.seed=3921835369594659663:-3219730304883530389
>>>    [junit] *** BEGIN
>>>
>>>org.apache.solr.request.NumFacetTermsFacetsTest.testNumFacetTermsFacetCou
>>>nt
>>> s: Insane FieldCache usage(s) ***
>>>    [junit] SUBREADER: Found caches for descendants of
>>> DirectoryReader(segments_3 _0(4.0):C6)+hgid_i1
>>>    [junit]     'DirectoryReader(segments_3 _0(4.0):C6)'=>'hgid_i1',class
>>>
>>>org.apache.lucene.search.FieldCache$DocTermsIndex,org.apache.lucene.searc
>>>h.
>>>
>>>cache.DocTermsIndexCreator@603bb3eb=>org.apache.lucene.search.cache.DocTe
>>>rm
>>> sIndexCreator$DocTermsIndexImpl#1026179434 (size =~ 372 bytes)
>>>    [junit]
>>>
>>>'org.apache.lucene.index.SegmentCoreReaders@7e8905bd'=>'hgid_i1',int,org.
>>>a
>>>
>>>pache.lucene.search.cache.IntValuesCreator@30781822=>org.apache.lucene.se
>>>ar
>>> ch.cache.CachedArray$IntValues#291172425 (size =~ 92 bytes)
>>>    [junit]
>>>    [junit] *** END
>>>
>>>org.apache.solr.request.NumFacetTermsFacetsTest.testNumFacetTermsFacetCou
>>>nt
>>> s: Insane FieldCache usage(s) ***
>>>
>>>
>>>
>>> On 7/1/11 6:54 AM, "Yonik Seeley" <yonik@lucidimagination.com> wrote:
>>>
>>>>On Thu, Jun 30, 2011 at 11:58 PM, Bill Bell <billnbell@gmail.com> wrote:
>>>>> I meant FC insanity. It does not appear to be an NPE.
>>>>
>>>>That's natural, and not a bug.  Grouping always uses per-segment field
>>>>cache entries, where faceting sometimes uses top level field caches.
>>>>
>>>>-Yonik
>>>>http://www.lucidimagination.com
>>>>
>>>>---------------------------------------------------------------------
>>>>To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>>For additional commands, e-mail: dev-help@lucene.apache.org
>>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message