lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Smiley <david.w.smi...@gmail.com>
Subject Re: Lucene GeoNear Search and Sort Performance
Date Thu, 20 Jul 2017 03:32:46 GMT
I like how you figured out how to add DocValues without having to modify
PointVectorStrategy at all -- nice.

But I think you are completely using PointVectorStrategy, even for
filtering (query)?  It does a relatively slow job at that; I suggested
using RecursivePrefixTreeStrategy for the filtering part.  RPT should use
less memory for filtering than PointVector.  (in theory; I haven't measured)

The memory shouldn't increase over time if you are not adding new documents
as well.  There ought to be a warm-up period up front but it ought the
memory use should stabilize.  Perhaps there is an issue in your approach to
measuring this.

On Tue, Jul 18, 2017 at 9:26 PM sc <scattamanchi@4info.com> wrote:

> David,
>
> I was able to get it working with minor changes in my codebase. I didn't
> have back port PointVectorStrategy class from 6.6.0 to 5.5.4
>
> Code:
>         final int maxNumberOfCachedQueries = 256;
>         final long maxRamBytesUsed = 50 * 1024L * 1024L; // 50MB
>
>         final QueryCache queryCache = new
> LRUQueryCache(maxNumberOfCachedQueries,
> maxRamBytesUsed);
>         final QueryCachingPolicy defaultCachingPolicy = new
> UsageTrackingQueryCachingPolicy();
>
>         this.strategy = new PointVectorStrategy(ctx, "pointvector");
>
> Indexing:
>                 doc.add(new DoubleDocValuesField("pointvector__x",
> longitude));
>                 doc.add(new DoubleDocValuesField("pointvector__y",
> latitude));
>
>                 Point pt = ctx.makePoint(longitude, latitude);
>                 for (IndexableField f :
> strategy.createIndexableFields(pt)) {
>                         doc.add(f);
>                 }
>
>                 doc.add(new StoredField(strategy.getFieldName(), pt.getX()
> + " " +
> pt.getY()));
>
> Searching :
>
>                 Point p = ctx.makePoint(lng, lat);
>                 Circle c = ctx.makeCircle(lng, lat, deg);
>                 SpatialArgs args = new
> SpatialArgs(SpatialOperation.Intersects, c);
>                 Query query = strategy.makeQuery(args);
>
>                 ValueSource valueSource =
> strategy.makeDistanceValueSource(p);
>                 Sort distSort = new
> Sort(valueSource.getSortField(false)).rewrite(searcher);
>
>                 int limit = 10;
>                 TopDocs topDocs = searcher.search(query, limit, distSort);
>
>
> While, I was able to get almost similar performance when compared with
> later
> versions of Lucene 6.6.0, but the memory consumption is more with Lucene
> 5.5.4 almost 5x.
>
> I am testing with 30K Lat/Lon and memory increases overtime.
>
> Any suggestions to improve memory consumption or code?
>
> Thanks
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Lucene-GeoNear-Search-and-Sort-Performance-tp4343468p4346726.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
> --
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message