lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brian Whitman (JIRA)" <j...@apache.org>
Subject [jira] Commented: (SOLR-303) Distributed Search over HTTP
Date Mon, 07 Jul 2008 22:16:31 GMT

    [ https://issues.apache.org/jira/browse/SOLR-303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12611376#action_12611376
] 

Brian Whitman commented on SOLR-303:
------------------------------------

Understood. Can I suggest a third alternative?

two new params: named &d.rows and &d.start with the implication that these get sent
unchanged to each of the shards. You may get back up to N*d.rows, where N is the # of shards.
That leaves the paging management up to the client.

Our use case is millions of documents across many shards, and we often do queries that are
"get all document of type X." There may be 5m type X documents. Doing a &rows=5000000
is unpredictable so we've previously done a loop of incrementing start by a 1000 and getting
1000 rows each time. But with this distributed setup, each successive batch query takes slightly
longer, and by the time we've gotten to the 5,001,000 batch queries are timing out and breaking
anyway. 





> Distributed Search over HTTP
> ----------------------------
>
>                 Key: SOLR-303
>                 URL: https://issues.apache.org/jira/browse/SOLR-303
>             Project: Solr
>          Issue Type: New Feature
>          Components: search
>            Reporter: Sharad Agarwal
>            Assignee: Yonik Seeley
>             Fix For: 1.3
>
>         Attachments: distributed.patch, distributed.patch, distributed.patch, distributed.patch,
distributed.patch, distributed.patch, distributed.patch, distributed.patch, distributed.patch,
distributed.patch, distributed.patch, distributed.patch, distributed_add_tests_for_intended_behavior.patch,
distributed_facet_count_bugfix.patch, distributed_pjaol.patch, fedsearch.patch, fedsearch.patch,
fedsearch.patch, fedsearch.patch, fedsearch.patch, fedsearch.patch, fedsearch.patch, fedsearch.stu.patch,
fedsearch.stu.patch, shards_qt.patch, solr-dist-faceting-non-ascii-all.patch
>
>
> Searching over multiple shards and aggregating results.
> Motivated by http://wiki.apache.org/solr/DistributedSearch

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message