lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Solr Streaming Queries Performance Issues [v7.2.1]
Date Fri, 28 Sep 2018 18:08:55 GMT
It Depends (tm). The behavior changed with Solr 7.5. Here are all the
gory details:

https://lucidworks.com/2017/10/13/segment-merging-deleted-documents-optimize-may-bad/

and for 7.5+
https://lucidworks.com/2018/06/20/solr-and-optimizing-your-index-take-ii/

Best,
Erick
On Fri, Sep 28, 2018 at 10:09 AM RAUNAK AGRAWAL
<agrawal.raunak@gmail.com> wrote:
>
> Hey Guys,
>
> This is the sample query I am making:
>
>
> curl http://localhost:8983/solr/collection_name/stream -d
> 'expr=facet(collection_name,q="id:953",bucketSorts="week
> desc",buckets="week",bucketSizeLimit=200,sum(sales),sum(amount),sum(days))'
>
>
> Also in my collection, I have almost 10 Billion documents with many
> deletions (close to 40%). I was planning to run optimise to merge the
> segments but spoke to admin team and lucidworks guys and they were against
> it saying that it will make very large segment file. Is it true that
> optimise in solr should not be used, as it comes with other issues?
>
> Thanks
>
> On Fri, Sep 28, 2018 at 7:40 AM Toke Eskildsen <toes@kb.dk> wrote:
>
> > On Thu, 2018-09-27 at 15:52 -0700, RAUNAK AGRAWAL wrote:
> > > But for last few days, we are observing now that streaming facet
> > > response is slower that json facets. Also we have increased the
> > > number of documents in collection (30%).
> >
> > Export performance goes down when segment size goes way up, so I would
> > expect streaming to do the same. I would not expect a 30% increase to
> > cause something serious on that account though. How many documents in
> > your index?
> >
> > - Toke Eskildsen, Royal Danish Library
> >
> >

Mime
View raw message