lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Upayavira ...@odoko.co.uk>
Subject Re: Is it possible to manually select a shard leader in a running SolrCloud?
Date Thu, 21 Feb 2013 21:43:39 GMT
Which of your three hosts did you point this request at?

Upayavira

On Thu, Feb 21, 2013, at 09:13 PM, Vaillancourt, Tim wrote:
> Correction, I used this curl:
> 
> curl -v
> 'http://<HOST>:8983/solr/admin/collections?action=CREATE&name=test&numShards=3&replicationFactor=2&maxShardsPerNode=2'
> 
> So 3 instances, 3 shards, 2 replicas per shard. ServerA becomes leader of
> all 3 shards in 4.1 with this call.
> 
> Tim Vaillancourt
> 
> -----Original Message-----
> From: Vaillancourt, Tim [mailto:TVaillancourt@ea.com] 
> Sent: Thursday, February 21, 2013 11:27 AM
> To: solr-user@lucene.apache.org; markrmiller@gmail.com
> Subject: RE: Is it possible to manually select a shard leader in a
> running SolrCloud?
> 
> Thanks Mark,
> 
> The real driver for me wanting to promote a different leader is when I
> create a new Collection via the Collections API across a multi-server
> SolrCloud, the leader of each shard is always the same host, so you're
> right that I'm tackling the wrong problem with this request, although it
> would fix it for me.
> 
> If I create the cores manually via the cores API, one-by-one, I am able
> to get what I expect, but when running this Collections API call on a 3
> SOLR 4.1 instance, 3 shard setup, 1 server becomes the leader of all 3
> shards, meaning it will get all the writes for everything (correct me if
> I am wrong). If so, this will not scale well with all writes to one node
> (or correct me if I am wrong)?
> 
> curl -v
> 'http://<HOST>:8983/solr/admin/collections?action=CREATE&name=test&numShards=3&replicationFactor=1&maxShardsPerNode=2'
> 
> Currently on my 3 instance SOLR 4.1 setup, the above call creates the
> following:
> 
> - ServerA is the leader of all 3 shards (the problem I want to address).
> - ServerB + ServerC are automagically replicas of the 3 leader shards on
> ServerA.
> 
> So again, my issue is one server gets all the writes. Does anyone else
> encounter this? If so, I should spawn a separate thread on my specific
> issue.
> 
> Cheers,
> 
> Tim
> 
> -----Original Message-----
> From: Mark Miller [mailto:markrmiller@gmail.com]
> Sent: Tuesday, February 19, 2013 8:44 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Is it possible to manually select a shard leader in a
> running SolrCloud?
> 
> You can't easily do it the way it's implemented in ZooKeeper. We would
> probably internally have to do the same thing - elect a new leader and
> drop him until the one we wanted came up. The main thing doing it
> internally would gain is that you could skip the elected guy from
> becoming the actual leader and just move on to the next candidate.
> Still some tricky corner cases to deal with and such as well.
> 
> I think for most things you would use this to solve, there is probably an
> alternate thing that should be addressed.
> 
> - Mark
> 
> On Mon, Feb 18, 2013 at 4:15 PM, Vaillancourt, Tim <TVaillancourt@ea.com>
> wrote:
> > Hey all,
> >
> > I feel having to unload the leader core to force an election is "hacky", and as
far as I know would still leave which node becomes the Leader to chance, ie I cannot guarantee
"NodeX" becomes Leader 100% in all cases.
> >
> > Also, this imposes additional load temporarily.
> >
> > Is there a way to force the winner of the Election, and if not, is there a known
feature-request for this?
> >
> > Cheers,
> >
> > Tim Vaillancourt
> >
> > -----Original Message-----
> > From: Joseph Dale [mailto:joey.dale@gmail.com]
> > Sent: Sunday, February 03, 2013 7:42 AM
> > To: solr-user@lucene.apache.org
> > Subject: Re: Is it possible to manually select a shard leader in a running SolrCloud?
> >
> > With solrclound all cores are collections. The collections API it just a wrapper
to call the core api a million times with one command.
> >
> > to /solr/admin/cores?action=CREATE&name=core1&collection=core1&shard=1
> >
> > Basically your "creating" the shard again, after leader props have gone out. Solr
will check ZK and find a core meeting that description, then simply get a copy of the index
from the leader of that shard.
> >
> >
> > On Feb 3, 2013, at 10:37 AM, Brett Hoerner <brett@bretthoerner.com> wrote:
> >
> >> What is the inverse I'd use to re-create/load a core on another 
> >> machine but make sure it's also "known" to SolrCloud/as a shard?
> >>
> >>
> >> On Sat, Feb 2, 2013 at 4:01 PM, Joseph Dale <joey.dale@gmail.com> wrote:
> >>
> >>>
> >>> To be more clear lets say bob it the leader of core 1. On bob do a 
> >>> /admin/cores?action=unload&name=core1. This removes the core/shard 
> >>> from bob, giving the other servers a chance to grab leader props.
> >>>
> >>> -Joey
> >>>
> >>> On Feb 2, 2013, at 11:27 AM, Brett Hoerner <brett@bretthoerner.com>
wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> I have a 5 server cluster running 1 collection with 20 shards,
> >>> replication
> >>>> factor of 2.
> >>>>
> >>>> Earlier this week I had to do a rolling restart across the cluster,

> >>>> this worked great and the cluster stayed up the whole time. The 
> >>>> problem is
> >>> that
> >>>> the last node I restarted is now the leader of 0 shards, and is 
> >>>> just holding replicas.
> >>>>
> >>>> I've noticed this node has abnormally high load average, while the 
> >>>> other nodes (who have the same number of shards, but more leaders 
> >>>> on
> >>>> average)
> >>> are
> >>>> fine.
> >>>>
> >>>> First, I'm wondering if that loud could be related to being a 5x 
> >>>> replica and 0x leader?
> >>>>
> >>>> Second, I was wondering if I could somehow flag single shards to
> >>> re-elect a
> >>>> leader (or force a leader) so that I could more evenly distribute 
> >>>> how
> >>> many
> >>>> leader shards each physical server has running?
> >>>>
> >>>> Thanks.
> >>>
> >>>
> >
> >
> 
> 
> 
> --
> - Mark
> 

Mime
View raw message