lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: field queries seem slow
Date Wed, 04 Nov 2009 19:07:15 GMT
By readers, I meant your searchers. Perhaps you were shutting
down your servers?

The warming isn't to pre-load authors, it's to pre-populate, particularly,
sort fields. Which are then kept in caches. There is considerable
overhead in loading the sort field the first time you sort by it. So,
my question was really based on the chance that "over the
weekend" corresponded to "the first queries after the server
restarted", or "the first query after the underlying index searchers
were (re)opened.

The real question comes down to whether the same form of query
(i.e. searching for different values on the same fields with the
same kind of sort) is slow all the time or just when things start up.

How fine is the resolution for your dates? Assuming that the sorting
is the issue, if you are storing dates in the millisecond range, that's
probably 20M dates that have to be loaded to sort. You might
want to think about a coarser resolution  if this has any relevance.

HTH
Erick

On Wed, Nov 4, 2009 at 1:54 PM, mike anderson <saidtherobot@gmail.com>wrote:

> Erik, we are doing a sort by date first, and then by score. I'm not sure
> what you mean by readers.
>
> Since we have nearly 6M authors attached to our 20M documents I'm not sure
> that autowarming would help that much (especially since we have very little
> overlap in what users are searching for). But maybe it would?
>
> Lance, I was just being a bit lazy. thanks though.
>
> -mike
>
>
> On Mon, Nov 2, 2009 at 10:27 PM, Lance Norskog <goksron@gmail.com> wrote:
>
> > This searches author:albert and (default text field): einstein. This
> > may not be what you expect?
> >
> > On Mon, Nov 2, 2009 at 2:30 PM, Erick Erickson <erickerickson@gmail.com>
> > wrote:
> > > Hmmmm, are you sorting? And has your readers been reopened? Is the
> > > second query of that sort also slow? If the answer to this last
> question
> > is
> > > "no",
> > > have you tried some autowarming queries?
> > >
> > > Best
> > > Erick
> > >
> > > On Mon, Nov 2, 2009 at 4:34 PM, mike anderson <saidtherobot@gmail.com
> > >wrote:
> > >
> > >> I took a look through my Solr logs this weekend and noticed that the
> > >> longest
> > >> queries were on particular fields, like "author:albert einstein". Is
> > this a
> > >> result consistent with other setups out there? If not, Is there a
> trick
> > to
> > >> make these go faster? I've read up on filter queries and use those
> when
> > >> applicable, but they don't really solve all my problems.
> > >>
> > >> If anybody wants to take a shot at it but needs to see my solrconfig,
> > etc
> > >> just let me know.
> > >>
> > >> Cheers,
> > >> Mike
> > >>
> > >
> >
> >
> >
> > --
> > Lance Norskog
> > goksron@gmail.com
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message