lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joel Bernstein (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SOLR-8965) Add Path reduce operation to aggregate paths taken in a session
Date Sun, 10 Apr 2016 16:06:25 GMT
Joel Bernstein created SOLR-8965:
------------------------------------

             Summary: Add Path reduce operation to aggregate paths taken in a session
                 Key: SOLR-8965
                 URL: https://issues.apache.org/jira/browse/SOLR-8965
             Project: Solr
          Issue Type: New Feature
            Reporter: Joel Bernstein


One of the things it's tricky to do at scale is session aggregation. MapReduce of course makes
this easy. Now that we have MapReduce it would be good to add some session aggregations to
the base library. 

The Path reduce operation can be used with the *reduce* function to concatenate the path taken
in a session into a single field. These path records can then be added to another SolrCloud
collection using the update stream. Once they have been consolidated in another collection
aggregations can be run on the paths using the RollupStream.

A HashRollupStream could also be developed to aggregate the paths as they are reduced. The
HashRollupStream would keep all the paths in a hash map during the aggregation so it would
not require the paths to be received in order.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message