lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Sturge (JIRA)" <>
Subject [jira] Commented: (SOLR-1709) Distributed Date Faceting
Date Mon, 08 Nov 2010 22:44:34 GMT


Peter Sturge commented on SOLR-1709:

Hi Peter,

Thanks for your message. There's of course the issue of 'now' as described in some of the
above comments. This is perhaps a little ancillary to this issue, but not totally irrelevant.

The issue of time zone/skew on distributed shards is currently handled by SOLR-1729 by passing
a '<epochtime>' parameter in the search query. This is then used by the
particapating shards to use as 'now'. Of course, there are a number of ways to skin that one,
but this is a straightforward solution that is backward compatible and still easy to implement
in client code.

Note that the change is not part of this patch - see SOLR-1729 for a separate
patch for this parameter. (kept separate because it's, strictly speaking, a separate issue
generally for distributed search)

It's not that eariler/later aren't supported - the date facet 'edges' are fine, it's just
the patch will 'quantize the ends' of the start/end date facets if the time is skewed from
the calling server. This is where SOLR-1729 comes into play, so that this doesn't happen.

   As this is a pre-3x/4x branch patch, the testing is a bit limited on the latest trunk(s).
Having said that, I have this (and SOLR-1729) building/running fine on my svn 3x branch release
Any other questions, or info you need, please do let me know.


> Distributed Date Faceting
> -------------------------
>                 Key: SOLR-1709
>                 URL:
>             Project: Solr
>          Issue Type: Improvement
>          Components: SearchComponents - other
>    Affects Versions: 1.4
>            Reporter: Peter Sturge
>            Priority: Minor
>         Attachments:,,,
> This patch is for adding support for date facets when using distributed searches.
> Date faceting across multiple machines exposes some time-based issues that anyone interested
in this behaviour should be aware of:
> Any time and/or time-zone differences are not accounted for in the patch (i.e. merged
date facets are at a time-of-day, not necessarily at a universal 'instant-in-time', unless
all shards are time-synced to the exact same time).
> The implementation uses the first encountered shard's facet_dates as the basis for subsequent
shards' data to be merged in.
> This means that if subsequent shards' facet_dates are skewed in relation to the first
by >1 'gap', these 'earlier' or 'later' facets will not be merged in.
> There are several reasons for this:
>   * Performance: It's faster to check facet_date lists against a single map's data, rather
than against each other, particularly if there are many shards
>   * If 'earlier' and/or 'later' facet_dates are added in, this will make the time range
larger than that which was requested
>         (e.g. a request for one hour's worth of facets could bring back 2, 3 or more
hours of data)
>     This could be dealt with if timezone and skew information was added, and the dates
were normalized.
> One possibility for adding such support is to [optionally] add 'timezone' and 'now' parameters
to the 'facet_dates' map. This would tell requesters what time and TZ the remote server thinks
it is, and so multiple shards' time data can be normalized.
> The patch affects 2 files in the Solr core:
> The main changes are in FacetComponent - ResponseBuilder is just to hold the completed
SimpleOrderedMap until the finishStage.
> One possible enhancement is to perhaps make this an optional parameter, but really, if parameters are specified, it is assumed they are desired.
> Comments & suggestions welcome.
> As a favour to ask, if anyone could take my 2 source files and create a PATCH file from
it, it would be greatly appreciated, as I'm having a bit of trouble with svn (don't shoot
me, but my environment is a Redmond-based os company).

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message