lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gabriele renzi <rff....@gmail.com>
Subject External ValueSource and value mapping
Date Tue, 11 May 2010 11:05:25 GMT
Hi everyone,

I am trying to implement something akin to  Solr's ExternalFileSource
backed by a Map object with plain lucene (as we are working on top of
an existing solution) and while it is easy to write a ValueSource that
does it I have a problem with the mapping phase.

Basically I have tried two things:
1. keep a map of  unique-ids for the documents, such as
 joe=10
 john = 20
and at runtime retrieve the unique-key field and use that to find the
value in the map
2. keep a map/array of _document ids_ as Solr's ExternalFileSource seems to do
 doc1=10
 doc2=20
and at runtime use the document id in floatValue as the lookup.

The problem I found is two-fold: the former solution seems to be
pretty slow, probably because of the need to fetch a Field for every
document involved in scoring, while the latter seems to be impossible:
as far as I can tell, when ValueSource.getValues is called different
index segments may be passed, meaning that the document id becomes a
non-unique key.
This also means I can't neithery precalculate this docIds/score
mapping, ena neither can I cache them using a hybrid solution.

 Looking at solr sources, this seems to be solved using
SolrIndexReader objects that have a #base attribute that can be used
to offset the document id, but as I said, we are using plain old
lucene's IndexReader objects and this seems impossible to replicate
using only them.

Is my assesent of the issue correct or am i missing something?
If it is, does someone have a solution for this, or has seen this
problem in the past and cares to share a workaround?

Thanks in advance.

-- 
blog en: http://www.riffraff.info
blog it: http://riffraff.blogsome.com

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message