lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Solr dynamic "on the fly fields"
Date Wed, 05 Jul 2017 16:08:54 GMT
Some aggregations are supported by combining stats with pivot facets? See:

https://lucidworks.com/2015/01/29/you-got-stats-in-my-facets/

Don't quite think that works for your use case though.

the other thing that _might_ help is all the Streaming
Expression/Streaming Aggregation work.

Best,
Erick


On Wed, Jul 5, 2017 at 6:23 AM, Pablo Anzorena <anzorena.fing@gmail.com> wrote:
> Thanks Erick for the answer. Function Queries are great, but for my use
> case what I really do is making aggregations (using Json Facet for example)
> with this functions.
>
> I have tried using Function Queries with Json Facet but it does not support
> it.
>
> Any other idea you can imagine?
>
>
>
>
>
> 2017-07-03 21:57 GMT-03:00 Erick Erickson <erickerickson@gmail.com>:
>
>> I don't know how one would do this. But I would ask what the use-case
>> is. Creating such fields at index time just seems like it would be
>> inviting abuse by creating a zillion fields as you have no control
>> over what gets created. I'm assuming your tenants don't talk to each
>> other....
>>
>> Have you thought about using function queries to pull this data out as
>> needed at _query_ time? See:
>> https://cwiki.apache.org/confluence/display/solr/Function+Queries
>>
>> Best,
>> Erick
>>
>> On Mon, Jul 3, 2017 at 12:06 PM, Pablo Anzorena <anzorena.fing@gmail.com>
>> wrote:
>> > Thanks Erick,
>> >
>> > For my use case it's not possible any of those solutions. I have a
>> > multitenancy scheme in the most basic level, that is I have a single
>> > collection with fields (clientId, field1, field2, ..., field50) attending
>> > many clients.
>> >
>> > Clients can create custom fields based on arithmetic operations of any
>> > other field.
>> >
>> > So, is it possible to update let's say field49 with the follow operation:
>> > log(field39) + field25 on clientId=43?
>> >
>> > Do field39 and field25 need to be stored to accomplish this? Is there any
>> > other way to avoid storing them?
>> >
>> > Thanks!
>> >
>> >
>> > 2017-07-03 15:00 GMT-03:00 Erick Erickson <erickerickson@gmail.com>:
>> >
>> >> There are two ways:
>> >> 1> define a dynamic field pattern, i.e.
>> >>
>> >> <dynamicField name="*_sum" type="float" ...../>
>> >>
>> >> Now just add any field in the doc you want. If it ends in "_sum" and
>> >> no other explicit field matches you have a new field.
>> >>
>> >> 2> Use the managed schema to add these on the fly. I don't recommend
>> >> this from what I know of your use case, this is primarily intended for
>> >> front-ends to be able to modify the schema and/or "field guessing".
>> >>
>> >> I do caution you though that either way don't go over-the-top. If
>> >> you're thinking of thousands of different fields that can lead to
>> >> performance issues.
>> >>
>> >> You can either put stuff in the field on your indexing client or
>> >> create a custom update component, perhaps the simplest would be a
>> >> "StatelessScriptUpdateProcessorFactory:
>> >>
>> >> see: https://cwiki.apache.org/confluence/display/solr/
>> >> Update+Request+Processors#UpdateRequestProcessors-
>> >> UpdateRequestProcessorFactories
>> >>
>> >> Best,
>> >> Erick
>> >>
>> >> On Mon, Jul 3, 2017 at 10:52 AM, Pablo Anzorena <
>> anzorena.fing@gmail.com>
>> >> wrote:
>> >> > Hey,
>> >> >
>> >> > I was wondering if there is some way to add fields "on the fly" based
>> on
>> >> > arithmetic operations on other fields. For example add a new field
>> >> > "custom_field" = log(field1) + field2 -5.
>> >> >
>> >> > Thanks.
>> >>
>>

Mime
View raw message