lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <ha.p...@arvatosystems.com>
Subject RE: Enrich search results with external data
Date Fri, 17 Apr 2015 22:02:19 GMT
Hi  Sujit,



Many thanks for your blog post, responding to my question, and suggesting the alternative
option ☺



I think I prefer your approach because we can supply our own Comparator. The reason is that
we need to meet some strict requirements: we can only call the external system once to retrieve
extra fields (price, inventory, etc.) for probably a subset of the search result.  Therefore
we need to be able to sort and facet on the list of items that some of them may not have external
fields. I think using the Comparator would help with the sorting but let me know if you have
different ideas.



Do you have suggestion how we should deal with the facet requirement? I am thinking about
adding another Facet Component that will be executed after the standard FacetComponent. Let
me know if you think we should consider other options.



Thanks,



-Ha



-----Original Message-----

From: sujitatgtalk@gmail.com [mailto:sujitatgtalk@gmail.com] On Behalf Of Sujit Pal

Sent: Saturday, April 11, 2015 10:23 AM

To: solr-user@lucene.apache.org; Ahmet Arslan

Subject: Re: Enrich search results with external data



Hi Ha,



I am the author of the blog post you mention. To your question, I don't know if the code will
work without change (since the Lucene/Solr API has evolved so much over the last few years),
but a more "preferred" way using Function Queries way may be found in slides for Timothy Potter's
talk here:

http://www.slideshare.net/thelabdude/boosting-documents-in-solr-lucene-revolution-2011



Here he speaks of external fields stored in a database and accessed using a custom component
(rather than from a flat file as in ExternalFieldField), and using function queries to influence
the ranking based on the external field. However, per this document on function queries, you
can use the output of a function query to sort as well by passing the function to the sort
parameter.

https://wiki.apache.org/solr/FunctionQuery#Sort_By_Function



Hope this helps,

Sujit





On Fri, Apr 10, 2015 at 10:38 PM, Ahmet Arslan <iorixxx@yahoo.com.invalid>

wrote:



> Hi,

>

> Who don't you include/add/index those additional fields, at least the

> one used in sorting?

>

> Also, you may find

> https://stanbol.apache.org/docs/trunk/components/enhancer/ relevant.

>

> Ahmet

>

>

>

> On Saturday, April 11, 2015 1:04 AM, "ha.pham@arvatosystems.com" <

> ha.pham@arvatosystems.com> wrote:

> This ticket seems to address the problem I have

>

> https://issues.apache.org/jira/browse/SOLR-1566

>

>

>

> and as the result of that ticket, DocTransformer is added since Solr 4.0.

> I wrote a simple DocTransformer and found that the transformer is

> executed AFTER pagination. In our application, we need the external

> fields added before sorting/pagination. I've looked around for the

> option to change the execution order but haven't had any luck. Does anyone know the solution?

>

>

>

> The ticket also states "it is not possible for components to add

> fields to outgoing documents which are not in the stored fields of the document".

> Does anyone know if this is still true?

>

>

>

> Thanks,

>

>

>

> -Ha

>

>

>

>

>

> -----Original Message-----

>

> From: Pham, Ha

>

> Sent: Thursday, April 09, 2015 11:41 PM

>

> To: solr-user@lucene.apache.org

>

>

> Subject: Enrich search results with external data

>

>

>

> Hi everyone,

>

>

>

> We have a requirement to append external data (e.g. price/inventory of

> product, retrieved from an ERP via web services) to query result and

> support sorting and pagination based on those external fields. For

> example if Solr returns 100 records and the page size user selects is

> 20, the sorting on the external  fields is still on 100 records. This

> limits us from enriching search results outside of Solr. I guess this

> is a common problem so hopefully someone could share their experience.

>

>

>

> I am considering using a PostFilter and enrich documents in collect()

> method as below

>

>

>

> @Override

>

> public void collect(int docId) throws IOException { DoubleField price

> = new DoubleField ("PRICE", 1.23, Field.Store.YES); Document

> currentDoc = context.reader().document(docId); currentDoc.add(price);

> }

>

>

>

> but the result documents don't have PRICE fields. Did I miss anything here?

>

>

>

> I also did some research and it seems the approach mentioned here

> http://sujitpal.blogspot.com/2011/05/custom-sorting-in-solr-using-exte

> rnal.html is close to what we need to achieve but since the document

> is 4 years old, I don't know if there's a better approach for our

> problem (we are using solr 5.0)?

>

>

>

> Thanks,

>

>

>

> -Ha

>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message