lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alejandro Calbazana <acalbaz...@gmail.com>
Subject Many Dynamic Fields + Indexing Strategy
Date Wed, 30 Oct 2013 01:37:31 GMT
Hi,

I have an application that has a fair number of dynamic fields in addition
to static fields.  The use case is that a customer can create any number of
dynamic fields and associate them with domain objects that we then pull
into an indexed document.  I have no way to know these fields in advance
and the expectation is that these fields are searchable using a field/value
query.  It is a multi-tenant environment and it is possible that there
could be a high volume of dynamic fields created.

My question is if there is a reasonable indexing strategy that can be used
to accommodate such a use case.  My concern is that I can end up with a
large number of dynamic fields which would bring querying and full indexing
to a slow down.  Through some testing, I've created unique dynamic fields
and got into the 50K - 100K range when my JVM began to behave poorly and go
OOM.  I understand why this happens but I'm interested in how to protect
against this.

My only thought at the moment is to split my single index into multiple
cores - one per tenant.  Has anyone else had this requirement?  How did you
handle it?

My schema is pretty much what I've described.  A handful of static fields
with the stock dynamic field pattern definitions.  I am using Solr 4.2.1.

Thanks,

Al

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message