Mailing-List: contact solr-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-dev@lucene.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-Id: <AF494086-11C6-4236-96C6-A31BFBE31387@apache.org>
From: Grant Ingersoll <gsingers@apache.org>
To: solr-dev@lucene.apache.org
In-Reply-To: <c68e39170907140912n7c1f556x4aa366066db71667@mail.gmail.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v935.3)
Subject: Re: SOLR automatic failover
Date: Wed, 15 Jul 2009 10:06:30 -0400
References: <85d3c3b60907131626j7ce281f1h3416ccb1e22e8bb2@mail.gmail.com>
 <5e76b0ad0907132128q796892e6kb6348fe5be086036@mail.gmail.com>
 <85d3c3b60907132240q4a7b035blfa8a047f70f5ee02@mail.gmail.com>
 <23F248CD-37AC-4712-B39F-41B9B8B5FBA5@opensourceconnections.com>
 <c68e39170907140912n7c1f556x4aa366066db71667@mail.gmail.com>

I've got some ZooKeeper work that I did on the plane the other day  
that I haven't had the time to write up just yet.

I've got the start for:
1. Automatic setting of Shards param for distributed search and new  
shards can automatically register themselves and be added to the mix  
(this is the furthest along, but is dependent on the firstSearcher/ 
newSearcher problem with distributed search that I mentioned in a  
diff. thread)
2. Start on way to bring replicants and masters on and offline and do  
rebalancing
3. RequestHandler based administration to add/delete/list/status  
ZooKeeper and it's information

I think the nice thing about all of it is it pretty much works  
seamlessly with existing Solr components and request handlers.  I need  
to bring the patch up to date and fill in a couple of pieces, but will  
post soon as a start.

-Grant

On Jul 14, 2009, at 12:12 PM, Yonik Seeley wrote:

> On Tue, Jul 14, 2009 at 11:49 AM, Eric
> Pugh<epugh@opensourceconnections.com> wrote:
>> Isn't the pool of slave Solr's behind a load balancer similar to SOLR
>> automatic failover?  Guess I am not sure what you mean?  Seems like  
>> what
>> ZooKeeper does
>> (http://hadoop.apache.org/zookeeper/docs/current/ 
>> zookeeperOver.html) is
>> analogous to the Solr master/pool of slave servers.
>
> Right now, the task of keeping track of what servers are in a cluster
> and what data they have is pushed off to the user.
> Zookeeper would keep track of that (and would also serve as a single
> point of configuration).
>
> Wouldn't it be nice to be able to start up another server like so:
> java -jar start.jar -Dcluster_id=foo
>
> And just have everything work (no updating loadbalancer configuration,
> other clients, etc).
>
> Then there is master failover - a somewhat harder problem... or very
> much harder, depending on how one approaches it.
>
> -Yonik
> http://www.lucidimagination.com

--------------------------
Grant Ingersoll
http://www.lucidimagination.com/

Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)  
using Solr/Lucene:
http://www.lucidimagination.com/search