lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (SOLR-1724) Real Basic Core Management with Zookeeper
Date Sat, 16 Jan 2010 18:57:54 GMT

    [ https://issues.apache.org/jira/browse/SOLR-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12801255#action_12801255
] 

Yonik Seeley commented on SOLR-1724:
------------------------------------

{quote}
These are discussed here: http://oss.101tec.com/jira/browse/KATTA-43

The basic design consideration is that failure of a node needs to automagically update the
ZK state accordingly. This allows all important updates to files to go one direction as well.
{quote}

We actually started out that way... (when a node went down there wasn't really any trace it
ever existed) but have been moving away from it.
ZK may not just be a reflection of the cluster but may also control certain aspects of the
cluster that you want persistent.  For example, marking a node as "disabled" (i.e. don't use
it).  One could create APIs on the node to enable and disable and have that reflected in ZK,
but it seems like more work than simply saying "change this znode".

> Real Basic Core Management with Zookeeper
> -----------------------------------------
>
>                 Key: SOLR-1724
>                 URL: https://issues.apache.org/jira/browse/SOLR-1724
>             Project: Solr
>          Issue Type: New Feature
>          Components: multicore
>    Affects Versions: 1.4
>            Reporter: Jason Rutherglen
>             Fix For: 1.5
>
>
> Though we're implementing cloud, I need something real soon I can
> play with and deploy. So this'll be a patch that only deploys
> new cores, and that's about it. The arch is real simple:
> On Zookeeper there'll be a directory that contains files that
> represent the state of the cores of a given set of servers which
> will look like the following:
> /production/cores-1.txt
> /production/cores-2.txt
> /production/core-host-1-actual.txt (ephemeral node per host)
> Where each core-N.txt file contains:
> hostname,corename,instanceDir,coredownloadpath
> coredownloadpath is a URL such as file://, http://, hftp://, hdfs://, ftp://, etc
> and
> core-host-actual.txt contains:
> hostname,corename,instanceDir,size
> Everytime a new core-N.txt file is added, the listening host
> finds it's entry in the list and begins the process of trying to
> match the entries. Upon completion, it updates it's
> /core-host-1-actual.txt file to it's completed state or logs an error.
> When all host actual files are written (without errors), then a
> new core-1-actual.txt file is written which can be picked up by
> another process that can create a new core proxy.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message