lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shai Erera (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-4769) Add a CountingFacetsAggregator which reads ordinals from a cache
Date Mon, 11 Feb 2013 22:05:12 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-4769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13576164#comment-13576164
] 

Shai Erera commented on LUCENE-4769:
------------------------------------

It's not like DirectPostingsFormat though. DPF hides the int[] from you, and you interact
with the general API, not knowing that under the covers it does things more efficiently. I
think that on LUCENE-4764, if we can prove that this specialization doesn't help much (i.e.
you don't need to cast to FacetsDV and pull the addresses and bytes), then it'd be compelling.
And if we had a DV type that had .get(doc, IntsRef), then an int[] DVFormat would also make
sense.

But if we implement that as a Codec, then the app would need to set both the Codec and the
matching FacetsAggregator. Also, it will be ineffective to use this Codec on existing large
indexes, as you won't gain anything. I treat this like FieldCache .. you have something indexed
one way, and read another way. Again, if there was a DVFormat that would let me ask for all
integers of a document, it'd be a different story I think.
                
> Add a CountingFacetsAggregator which reads ordinals from a cache
> ----------------------------------------------------------------
>
>                 Key: LUCENE-4769
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4769
>             Project: Lucene - Core
>          Issue Type: New Feature
>          Components: modules/facet
>            Reporter: Shai Erera
>            Assignee: Shai Erera
>         Attachments: LUCENE-4769.patch
>
>
> Mike wrote a prototype of a FacetsCollector which reads ordinals from a CachedInts structure
on LUCENE-4609. I ported it to the new facets API, as a FacetsAggregator. I think we should
offer users the means to use such a cache, even if it consumes more RAM. Mike tests show that
this cache consumed x2 more RAM than if the DocValues were loaded into memory in their raw
form. Also, a PackedInts version of such cache took almost the same amount of RAM as straight
int[], but the gains were minor.
> I will post the patch shortly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message