lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Noble Paul (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (SOLR-561) Solr replication by Solr (for windows also)
Date Wed, 09 Jul 2008 11:39:31 GMT

    [ https://issues.apache.org/jira/browse/SOLR-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12611989#action_12611989
] 

noble.paul edited comment on SOLR-561 at 7/9/08 4:37 AM:
---------------------------------------------------------

This patch relies on the IndexDeletionPolicy to identify files to be replicated. It also supports
replication of conf files. No need to register any listeners/ QueryResponseWriters

The configuration is as follows
on master 
{code:title=solrconfig.xml}
<requestHandler name="/replication" class="solr.ReplicationHandler" >
    <lst name="master">
        <!--Replicate on 'optimize' it can also be  'commit' -->
        <str name="replicateAfter">commit</str>
        <!--Config files to be to be replicated-->
         <str name="confFiles">schema.xml,stopwords.txt,elevate.xml</str>    
     
    </lst>
</requestHandler>
{code}
on slave
{code:title=solrconfig.xml}
<requestHandler name="/replication" class="solr.ReplicationHandler" >
    <lst name="slave">
        <str name="masterUrl">http://localhost:port/solr/corename/replication</str>
 
        <str name="pollInterval">00:00:20</str>  
     </lst>
</requestHandler>
{code}

The Replication strategy is changed as follows
* CMD_INDEX_VERSION: (command=indexversion)gets the version of the current _IndexCommit_ to
be replicated from the master. if the version is same, no need to replicate. If it is different
* CMD_FILE_LIST : (command=filelist)Get the list of file names for the current _IndexCommit_
. Checks with the local index and identifies modified files by comparing names an sizes. It
also returns the details of the conf files
* CMD-FILE_CONTENT : (command=filecontent)For each files to be downloaded, issue this command
an download the content to a temp folder. After successful completion copy them to the index
folder  and isse a commit
* If the current index is stale, or not able to synchronize, copy all the files . An _index.properties_
file is written, which has the location of the new index directory
* _CoreDescriptor_ has a new method to reload core.
* If conf files are modified they are copied to the conf folder after taking a backup of the
old. Then the core is reloaded








      was (Author: noble.paul):
    This patch relies on the IndexDeletionPolicy to identify files to be replicated. It also
supposrts replication of conf files. No need to register any listeners/ QueryResponseWriters

The configuration is as follows
on master 
{code:title=solrconfig.xml}
<requestHandler name="/replication" class="solr.ReplicationHandler" >
    <lst name="master">
        <!--Replicate on 'optimize' it can also be  'commit' -->
        <str name="replicateAfter">commit</str>
        <!--Config files to be to be replicated-->
         <str name="confFiles">schema.xml,stopwords.txt,elevate.xml</str>    
     
    </lst>
</requestHandler>
{code}
on slave
{code:title=solrconfig.xml}
<requestHandler name="/replication" class="solr.ReplicationHandler" >
    <lst name="slave">
        <str name="masterUrl">http://localhost:port/solr/corename/replication</str>
 
        <str name="pollInterval">00:00:20</str>  
     </lst>
</requestHandler>
{code}

The Replication strategy is changed as follows
* CMD_INDEX_VERSION: gets the version of the current _IndexCommit_ to be replicated from the
master. if the version is same, no need to replicate. If it is different
* CMD_FILE_LIST : Get the list of file names for the current _IndexCommit_ . Checks with the
local index and identifies modified files by comparing names an sizes. 
* CMD-FILE_CONTENT : For each files to be downloaded, issue this command an download the content
to a temp folder. After successful completion copy them to the index folder  and isse a commit
* If the current index is stale, or not able to synchronize, copy all the files . An index,properties
file is written, which has the location of the new index directory
* _CoreDescriptor_ has a new method to reload core.
* If conf files are modified they are copied to the conf folder after taking a backup of the
old. Then the core is reloaded







  
> Solr replication by Solr (for windows also)
> -------------------------------------------
>
>                 Key: SOLR-561
>                 URL: https://issues.apache.org/jira/browse/SOLR-561
>             Project: Solr
>          Issue Type: New Feature
>          Components: replication
>    Affects Versions: 1.3
>         Environment: All
>            Reporter: Noble Paul
>         Attachments: deletion_policy.patch, SOLR-561.patch, SOLR-561.patch, SOLR-561.patch
>
>
> The current replication strategy in solr involves shell scripts . The following are the
drawbacks with the approach
> *  It does not work with windows
> * Replication works as a separate piece not integrated with solr.
> * Cannot control replication from solr admin/JMX
> * Each operation requires manual telnet to the host
> Doing the replication in java has the following advantages
> * Platform independence
> * Manual steps can be completely eliminated. Everything can be driven from solrconfig.xml
.
> ** Adding the url of the master in the slaves should be good enough to enable replication.
Other things like frequency of
> snapshoot/snappull can also be configured . All other information can be automatically
obtained.
> * Start/stop can be triggered from solr/admin or JMX
> * Can get the status/progress while replication is going on. It can also abort an ongoing
replication
> * No need to have a login into the machine 
> This issue can track the implementation of solr replication in java

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message