lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-2855) Large number of range facets causes server to lock up
Date Tue, 29 Nov 2011 01:45:40 GMT

    [ https://issues.apache.org/jira/browse/SOLR-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159023#comment-13159023
] 

Hoss Man commented on SOLR-2855:
--------------------------------

bq. The problem with start+gap+end is, that they are "easy" to get wrong, compared to, let's
say, start+end+number_of_gaps. 

Hmmm... i'm not sure why that would be true - if your client code can accidentally compute
& request facet.range.start=$EXTREMELY_LOW_NUM then couldn't someone else's client code
just as easily compute & request facet.range.numOfGaps=$EXTREMELY_BIG_NUM ?

bq. I guess the latter is a more common use case anyway, and would save some calculation that
can go haywire on overflows. Using static or stored numbers is always less error prone than
calculated ones.

My experiences is quite the opposite: Most use cases i've seen for range faceting wouldn't
benefit from a static number of gaps request -- particularly when start & end are specified
at run time, because it usually makes the size of the gaps very arbitrary and not very meaningful
to people (ie: if i'm searching for xmas gifts between $20 and $75 i'd rather see a price
range facet in $10 increments (with the last one being $5) then see 10 even sized ranges of
$5.50 each).  Besides which: specifying the gap (instead of the number of gaps) is the only
way date faceting is of any use at all.

(I'd love to hear about more use cases where specifying start+end+numBuckets would be advantageous,
but that really seems like it should be on the solr-user list, or in a new feature request
issue)

If you really want direct control over teh number of gaps, you may want to track SOLR-2366
-- there's been a lot of hashing/trashing about what the API should look like, but at a high
level it's aiming for a feature where you specify the exact endpoints of the ranges you want
in a succinct manner.
                
> Large number of range facets causes server to lock up
> -----------------------------------------------------
>
>                 Key: SOLR-2855
>                 URL: https://issues.apache.org/jira/browse/SOLR-2855
>             Project: Solr
>          Issue Type: Bug
>          Components: SearchComponents - other
>    Affects Versions: 3.4
>            Reporter: Bernhard Frauendienst
>              Labels: facet, range
>
> (NOTE: I'm aware that this is not exactly a BUG in the sense that something is not working
correctly, but it is still something that might be considered to be changed.)
> A bug in our code triggered an integer overflow which caused very large range facets
to be requested on a TrieIntField
> (e.g. {{facet.range.start=*-2147483648*&facet.range.end=1000&facet.range.gap=1}})
> This caused the Solr server to allocate huge amounts of memory so that it soon had filled
up the whole 18 GB the JVM had available, and the Garbage Collector wasn't able to free it
(fast enough?) and was thus taking up nearly all of the CPU, which finally led to a complete
Denial of Service.
> While the cause for this behavior obviously lies within the buggy code, it might still
be desirable for Solr not to lock up for "wrong" parameters but throw an error instead.
> Throwing an exception if more ranges than the {{facet.limit}} value (or a similar, new
parameter) for this field would be generated might be a good solution.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message