lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikhail Khludnev <m...@apache.org>
Subject Huge Facets and Streaming
Date Mon, 21 Aug 2017 10:01:39 GMT
Hello!

I need to count really wide facet on 30 shards index with roughly 100M
docs, the facet response is about 100M values takes 0.5G in text file.

So, far I experimented with old facets. It calculates per shard facets
fine, but then a node which attempts to merge such 30 responses fails due
to OOM. It's reasonable.

I suppose I'll get pretty much same with json.facet, or it's better
scalable?

I want to experiment with Streaming Expression, which I've never taken yet.
I've found facet() expression and select() with partitionKeys they'll try
to merge facet values in FacetComponent/Module anyway.
Is there a way to merge per-shard facet responses with Streaming?

-- 
Sincerely yours
Mikhail Khludnev

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message