lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: Near Real Time + Facets + Hierarchical Faceting (Pivot Table) with Date Range: huge data set
Date Tue, 14 Aug 2012 02:16:48 GMT
There is a per segment faceting option - but I think just for single value
fields right now?


On Mon, Aug 13, 2012 at 2:38 PM, Fuad Efendi <fuad@efendi.ca> wrote:

> SOLR-4.0
>
> I am trying to implement this; funny idea to share:
>
> 1. http://wiki.apache.org/solr/HierarchicalFaceting
> unfortunately it does not support date ranges. However, workaround: use
> "String" type instead of "*_tdt" and define fields such as
> published_hour
> published_day
> published_week
> Ċ 
>
> Of course you will need to stick with timezone; but you can add an
> index(es)
> for each timezone. And most important, "string" facets are much faster than
> "Date Trie" ranges.
>
>
>
> 2. Our index is overs 100 millions (from social networks) and rapidly grows
> (millions a day); cache warm up takes few minutes; Near-Real-Time does not
> work with faceting.
>
> HoweverĊ  another workaround: we can have Daily Core (optimized at
> midnight),
> plus Current Core (only today's data, optimized), plus Last Hour Core (near
> real time)
>
> "Last Hour Data" is small enough and we can use Facets with Near Real Time
> feature
>
> Service layer will accumulate search results from three layers, it will be
> near real time.
>
>
>
> Any thoughts? Thanks,
>
>
>
>
> --
> Fuad Efendi
> 416-993-2060
> Tokenizer Inc., Canada
> http://www.tokenizer.ca
> http://www.linkedin.com/in/lucene
>
>
>
>


-- 
- Mark

http://www.lucidimagination.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message