lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Norskog <goks...@gmail.com>
Subject Re: priority queue in query component
Date Tue, 09 Feb 2010 03:36:04 GMT
At this point, Distributed Search does not support any recovery if
when one or more shards fail. If any fail or time out, the whole query
fails.

On Sat, Feb 6, 2010 at 9:34 AM, mike anderson <saidtherobot@gmail.com> wrote:
> "so if we received the response from shard2 before shard1, we would just
> queue it up and wait for the response to shard1."
>
> This crossed my mind, but my concern was how to handle the case when shard1
> never responds. Is this something I need to worry about?
>
> -mike
>
> On Sat, Feb 6, 2010 at 11:33 AM, Yonik Seeley <yonik@lucidimagination.com>wrote:
>
>> It seems like changing an element in a priority queue breaks the
>> invariants, and hence it's not doable with a priority queue and with
>> the current strategy of adding sub-responses as they are received.
>>
>> One way to continue using a priority queue would be to add
>> sub-responses to the queue in the preferred order... so if we received
>> the response from shard2 before shard1, we would just queue it up and
>> wait for the response to shard1.
>>
>> -Yonik
>> http://www.lucidimagination.com
>>
>>
>> On Sat, Feb 6, 2010 at 10:35 AM, mike anderson <saidtherobot@gmail.com>
>> wrote:
>> > I have a need to favor documents from one shard over another when
>> duplicates
>> > occur. I found this code in the query component:
>> >
>> >          String prevShard = uniqueDoc.put(id, srsp.getShard());
>> >          if (prevShard != null) {
>> >            // duplicate detected
>> >            numFound--;
>> >
>> >            // For now, just always use the first encountered since we
>> can't
>> > currently
>> >            // remove the previous one added to the priority queue.  If
we
>> > switched
>> >            // to the Java5 PriorityQueue, this would be easier.
>> >            continue;
>> >            // make which duplicate is used deterministic based on shard
>> >            // if (prevShard.compareTo(srsp.shard) >= 0) {
>> >            //  TODO: remove previous from priority queue
>> >            //  continue;
>> >            // }
>> >          }
>> >
>> >
>> > Is there a ticket open for this issue? What would it take to fix?
>> >
>> > Thanks,
>> > Mike
>> >
>>
>



-- 
Lance Norskog
goksron@gmail.com

Mime
View raw message