lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Upayavira ...@odoko.co.uk>
Subject Re: Custom Solr caches in a FunctionQuery that emulates the ExternalFileField
Date Fri, 10 Jul 2015 19:25:02 GMT
Hi Erick,

You are right that I could actually be asking for a stored field. That's
an exceptionally good point, and yes, would suck. Better would be to
retrieve a docValue from document. I'll look into that.

Upayavira

On Fri, Jul 10, 2015, at 06:28 PM, Erick Erickson wrote:
> Upayavira:
> 
> bq: retrieve the value of field Y for that doc
> 
> If this is fetching the stored field it's going to be horrible as
> it'll probably read/decompress a 16K block each time. Yuuuuccck. If
> you can read the value from a DocValues field (or, indeed, any indexed
> field which would only really work for non-text types).
> 
> There's also Solr's User cache. This is just a cache like filterCache
> etc. that you code up yourself. The kicker is that it gets a "refresh
> yourself" message whenever a new searcher is opened.
> 
> I vaguely remember some work about efficiently finding the <uniqueKey>
> bit can't lay my hands on it.
> 
> I also wonder which of these would be handled by updateable doc values
> and whether that effort is more general?
> 
> Best,
> Erick
> 
> On Fri, Jul 10, 2015 at 8:30 AM, Upayavira <uv@odoko.co.uk> wrote:
> > Mikhail,
> >
> > Thanks for pointing this out.
> >
> > I'd say that ticket is in distinct need of some examples or use-cases.
> > It is extremely hard to work out what "scoring" actually means. What is
> > used to score what?
> >
> > It'd be great to see some examples and some explanations as to what
> > effect those examples have on scoring.
> >
> > I'll dig into that patch to see if I can work it out.
> >
> > Upayavira
> >
> >
> > On Fri, Jul 10, 2015, at 04:15 PM, Mikhail Khludnev wrote:
> >> I've heard that people use
> >> https://issues.apache.org/jira/browse/SOLR-6234
> >> for such purpose - adding scores from fast moving core to the bigger slow
> >> moving one
> >>
> >> On Fri, Jul 10, 2015 at 4:54 PM, Upayavira <uv@odoko.co.uk> wrote:
> >>
> >> > All,
> >> >
> >> > I have knocked up what I think could be a really cool function query -
> >> > it allows you to retrieve a value from another core (much like a pseudo
> >> > join) and use that value during scoring (much like an
> >> > ExternalFileField).
> >> >
> >> > Examples:
> >> >  * Selective boosting of documents based upon a category based value
> >> >  * boost on aggregated popularity values
> >> >  * boost on fast moving data on your slow moving index
> >> >
> >> > It *works* but it does so very slowly (on 3m docs, milliseconds without,
> >> > and 24s with it). There are two things that happen a lot:
> >> >
> >> >  * locate a document with unique ID value of X
> >> >  * retrieve the value of field Y for that doc
> >> >
> >> > What it seems to me now is that I need to implement a cache that will
> >> > have a string value as the key and the (float) field value as the
> >> > object, that is warmed alongside existing caches.
> >> >
> >> > Any pointers to examples of how I could do this, or other ways to do the
> >> > conversion from a key value to a float value faster?
> >> >
> >> > NB. I hope to contribute this if I can make it perform.
> >> >
> >> > Thanks!
> >> >
> >> > Upayavira
> >> >
> >>
> >>
> >>
> >> --
> >> Sincerely yours
> >> Mikhail Khludnev
> >> Principal Engineer,
> >> Grid Dynamics
> >>
> >> <http://www.griddynamics.com>
> >> <mkhludnev@griddynamics.com>

Mime
View raw message