lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Per Steffensen (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SOLR-4114) Collection API: Allow multiple shards from one collection on the same Solr server
Date Wed, 28 Nov 2012 14:55:01 GMT

    [ https://issues.apache.org/jira/browse/SOLR-4114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505506#comment-13505506
] 

Per Steffensen edited comment on SOLR-4114 at 11/28/12 2:54 PM:
----------------------------------------------------------------

Another more urgent problem (for me) is that I need to do another change to the Solr Collection
API, before we can use it as a replacement for what we already do in our project (where we
create each shard one by one in OUR code). We split our set of Solr servers into two subsets
- Data-Solrs and Search-Solrs. The Search-Solrs are not supposed to carry any data and therefore
to be occupied by indexing. Search-Solr instead play the role of receiving queries from the
outside, sub-quering the Data-Solrs and combining the final total response to the "outside".
Data-Solrs are where we create the data-carrying collections. Data-Solrs need more CPU and
IO-capabilities while Search-Solrs need more RAM - hence the splitup.

Therefore I need to be able to provide a list of Solrs to the create operation of the Solr
Collection API. The shards of the collection to be created are then only allowed to be spread
over the Solrs in this list - default list could be "all Solrs". As this list we, in our Solr-based
projbect, will give our list of Data-Solrs.

Can I add such a feature to this SOLR-4114 and include it in a combined patch, or do you prefer
another ticket for this change? I can create another issue but provide a combined patch. Are
you interrested in such a feature at all? That is, a feature where the create operation takes
a list of Solrs to spread the created shards over.
                
      was (Author: steff1193):
    Another more urgent problem (for me) is that I need to do another change to the Solr Collection
API, before we can use it as a replacement for what we already do in our project (where we
create each shard one by one in OUR code). We split our set of Solr servers into two subsets
- Data-Solrs and Search-Solrs. The Search-Solrs are not supposed to carry any data and therefore
to be occupied by indexing. Search-Solr instead play the role of receiving queries from the
outside, sub-quering the Data-Solrs and combining the final total response to the "outside".
Data-Solrs are where we create the data-carrying collections. Data-Solrs need more CPU and
IO-capabilities while Search-Solrs need more RAM - hence the splitup.

Therefore I need to be able to provide a list of Solrs to the create operation of the Solr
Collection API. The shards are then only allowed to be spread shards for the collection over
the Solrs in this list - default list could be "all Solrs". As this list we, in our Solr-based
projbect, will give our list of Data-Solrs.

Can I add such a feature to this SOLR-4114 and include it in a combined patch, or do you prefer
another ticket for this change? I can create another issue but provide a combined patch. Are
you interrested in such a feature at all? That is, a feature where the create operation takes
a list of Solrs to spread the created shards over.
                  
> Collection API: Allow multiple shards from one collection on the same Solr server
> ---------------------------------------------------------------------------------
>
>                 Key: SOLR-4114
>                 URL: https://issues.apache.org/jira/browse/SOLR-4114
>             Project: Solr
>          Issue Type: New Feature
>          Components: multicore, SolrCloud
>    Affects Versions: 4.0
>         Environment: Solr 4.0.0 release
>            Reporter: Per Steffensen
>            Assignee: Per Steffensen
>              Labels: collection-api, multicore, shard, shard-allocation
>         Attachments: SOLR-4114.patch
>
>
> We should support running multiple shards from one collection on the same Solr server
- the run a collection with 8 shards on a 4 Solr server cluster (each Solr server running
2 shards).
> Performance tests at our side has shown that this is a good idea, and it is also a good
idea for easy elasticity later on - it is much easier to move an entire existing shards from
one Solr server to another one that just joined the cluter than it is to split an exsiting
shard among the Solr that used to run it and the new Solr.
> See dev mailing list discussion "Multiple shards for one collection on the same Solr
server"

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message