lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lisa Carter (JIRA)" <j...@apache.org>
Subject [jira] Commented: (SOLR-534) Return all query results with parameter rows=-1
Date Wed, 06 Jan 2010 04:31:55 GMT

    [ https://issues.apache.org/jira/browse/SOLR-534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796982#action_12796982
] 

Lisa Carter commented on SOLR-534:
----------------------------------

I would argue that REALLY_BIG_NUMBER is actually significantly MORE dangerous than a crash.


Here's why: A crash at least lets the programmer know something went wrong. Missing data is
a silent failure. 

1) If the result set is too large for the client, it will run out of memory and generate an
exception. The programmer will immediately know they did something wrong.

2) If the result set is too large for the network (unlikely) this will disconnect and fail.
The programmer will immediately know they did something wrong.

3) If the result set is too large for solr, solr should not crash but rather return a page
with the standard error handler "result set too large"/"out of memory". The programmer will
immediately know they did something wrong. Solr sure as heck better be checking this already--you
never know when you'll run into bizarre low memory conditions;allocations should ALWAYS be
checked for.

But if you use the REALLY_BIG_NUMBER approach, the same bad programmer who never thought he
would get back more than a 1000 records will never check whether the result set contains more
than 1000 records either. If the programmer was expecting the complete result set and the
database now contains 1002 records instead of 999, they will not know there is a problem...
the last records in the set are simply truncated. The programmer who wrote the code may not
be the person maintaining the application, quite common in production environments. The maintenance
person may not know for weeks or months that a problem even exists! 

The -1 approach ensures immediate, loud failure.

The REALLY_BIG_NUMBER ensures only silent failure.

While it's impossible to idiot-proof everything, loud failure is always preferable to silent
failure. Barking loudly saves the poor soul who maintains the idiot's code a lot of heartache.


> Return all query results with parameter rows=-1
> -----------------------------------------------
>
>                 Key: SOLR-534
>                 URL: https://issues.apache.org/jira/browse/SOLR-534
>             Project: Solr
>          Issue Type: New Feature
>          Components: search
>    Affects Versions: 1.3
>         Environment: Tomcat 5.5
>            Reporter: Lars Kotthoff
>            Priority: Minor
>         Attachments: solr-all-results.patch
>
>
> The searcher should return all results matching a query when the parameter rows=-1 is
given.
> I know that it is a bad idea to do this in general, but as it explicitly requires a special
parameter, people using this feature will be aware of what they are doing. The main use case
for this feature is probably debugging, but in some cases one might actually need to retrieve
all results because they e.g. are to be merged with results from different sources.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message