lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-2548) Multithreaded faceting
Date Tue, 14 Jun 2011 19:04:47 GMT

    [ https://issues.apache.org/jira/browse/SOLR-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049341#comment-13049341
] 

Hoss Man commented on SOLR-2548:
--------------------------------

Janne: thanks for the awesome patch!

in general i think this type of functionality is a good idea -- the real question is how it
should be configured/controlled.

admins with many CPUs who expect low amounts of concurrent user traffic might be ok with spawning
availableProcessors() threads per request, but admins with less CPUs then concurrent requests
are going to prefer that individual requests stay single threaded and take a little longer.

The suggestion to use a thread pool based executorservice definitely seems like it makes more
sense, then it just becomes a matter of asking the admin to configure a simple number determining
the size of the threadpool, we just need to support a few sentinal values: NUM_CPUS, and NONE
(always use callers thread).

since we'd want a threadpool that lives longer then a single request, this definitely shouldn't
be an option specified via SolrParams (not to mention the risk involved if people don't lock
it down with invariants).  That leaves the question of wether this should be an "init" param
on the FacetComponent, or something more global.

My first thought was that we should bite the bullet and add a new top level config for a global
thread pool executor service that any solr plugin could start using, but after thinking about
it some more i think that would not only be premature, but perhaps even a bad idea in general
-- even if we assume something like DIH, UIMA, or Highlighting could also take advantage of
a shared thread pool owned by solr, if you really care about parallelizing faceting, you probably
wouldn't want some other intensive component starving out the thread pool (or vice versa)

> Multithreaded faceting
> ----------------------
>
>                 Key: SOLR-2548
>                 URL: https://issues.apache.org/jira/browse/SOLR-2548
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 3.1
>            Reporter: Janne Majaranta
>            Priority: Minor
>              Labels: facet
>         Attachments: SOLR-2548.patch, SOLR-2548_for_31x.patch
>
>
> Add multithreading support for faceting.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message