lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lance Norskog (JIRA)" <j...@apache.org>
Subject [jira] Commented: (SOLR-1383) Replication causes master to fail to delete old index files
Date Mon, 31 Aug 2009 20:25:32 GMT

    [ https://issues.apache.org/jira/browse/SOLR-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749609#action_12749609
] 

Lance Norskog commented on SOLR-1383:
-------------------------------------

This change to the commitReserveDuration did make a difference.

In one test run, it did exactly what we want it to: the master & slave data directories
had matching and current indexes.

In another test run, the same configuration failed in a new way: the master directory looked
exactly as it should, but the slave data directory wound up with two index snapshot folders.
They contained completely separate copies of the same index. There was no solr/data/index
folder, only the two snapshots. I did not remember to get the contents of the index.properties
and replication.properties files; mea culpa.  

Here is a full listing under the slave solr/data directory:
{code}
ls -l /index/slave/solr/data/*
-rw-r--r-- 1 root root   75 Aug 31 15:12 /index/slave/solr/data/index.properties
-rw-r--r-- 1 root root  107 Aug 31 15:12 /index/slave/solr/data/replication.properties

/index/slave/solr/data/index.20090829060632:
total 120
-rw-r--r-- 1 root root 4542 Aug 29 18:06 _a.fdt
-rw-r--r-- 1 root root  132 Aug 29 18:06 _a.fdx
-rw-r--r-- 1 root root  150 Aug 29 18:06 _a.fnm
-rw-r--r-- 1 root root 1369 Aug 29 18:06 _a.frq
-rw-r--r-- 1 root root   84 Aug 29 18:06 _a.nrm
-rw-r--r-- 1 root root 1842 Aug 29 18:06 _a.prx
-rw-r--r-- 1 root root  112 Aug 29 18:06 _a.tii
-rw-r--r-- 1 root root 7158 Aug 29 18:06 _a.tis
-rw-r--r-- 1 root root   23 Aug 29 18:06 _a.tvd
-rw-r--r-- 1 root root  149 Aug 29 18:06 _a.tvf
-rw-r--r-- 1 root root  260 Aug 29 18:06 _a.tvx
-rw-r--r-- 1 root root  305 Aug 29 18:06 _b.fdt
-rw-r--r-- 1 root root   12 Aug 29 18:06 _b.fdx
-rw-r--r-- 1 root root   71 Aug 29 18:06 _b.fnm
-rw-r--r-- 1 root root   64 Aug 29 18:06 _b.frq
-rw-r--r-- 1 root root    7 Aug 29 18:06 _b.nrm
-rw-r--r-- 1 root root   90 Aug 29 18:06 _b.prx
-rw-r--r-- 1 root root   35 Aug 29 18:06 _b.tii
-rw-r--r-- 1 root root  697 Aug 29 18:06 _b.tis
-rw-r--r-- 1 root root  648 Aug 29 18:06 _c.fdt
-rw-r--r-- 1 root root   20 Aug 29 18:06 _c.fdx
-rw-r--r-- 1 root root  111 Aug 29 18:06 _c.fnm
-rw-r--r-- 1 root root  244 Aug 29 18:06 _c.frq
-rw-r--r-- 1 root root   10 Aug 29 18:06 _c.nrm
-rw-r--r-- 1 root root  366 Aug 29 18:06 _c.prx
-rw-r--r-- 1 root root   51 Aug 29 18:06 _c.tii
-rw-r--r-- 1 root root 1504 Aug 29 18:06 _c.tis
-rw-r--r-- 1 root root  700 Aug 29 18:06 segments_d

/index/slave/solr/data/index.20090831031246:
total 120
-rw-r--r-- 1 root root 4542 Aug 31 15:12 _a.fdt
-rw-r--r-- 1 root root  132 Aug 31 15:12 _a.fdx
-rw-r--r-- 1 root root  150 Aug 31 15:12 _a.fnm
-rw-r--r-- 1 root root 1369 Aug 31 15:12 _a.frq
-rw-r--r-- 1 root root   84 Aug 31 15:12 _a.nrm
-rw-r--r-- 1 root root 1842 Aug 31 15:12 _a.prx
-rw-r--r-- 1 root root  112 Aug 31 15:12 _a.tii
-rw-r--r-- 1 root root 7158 Aug 31 15:12 _a.tis
-rw-r--r-- 1 root root   23 Aug 31 15:12 _a.tvd
-rw-r--r-- 1 root root  149 Aug 31 15:12 _a.tvf
-rw-r--r-- 1 root root  260 Aug 31 15:12 _a.tvx
-rw-r--r-- 1 root root  305 Aug 31 15:12 _b.fdt
-rw-r--r-- 1 root root   12 Aug 31 15:12 _b.fdx
-rw-r--r-- 1 root root   71 Aug 31 15:12 _b.fnm
-rw-r--r-- 1 root root   64 Aug 31 15:12 _b.frq
-rw-r--r-- 1 root root    7 Aug 31 15:12 _b.nrm
-rw-r--r-- 1 root root   90 Aug 31 15:12 _b.prx
-rw-r--r-- 1 root root   35 Aug 31 15:12 _b.tii
-rw-r--r-- 1 root root  697 Aug 31 15:12 _b.tis
-rw-r--r-- 1 root root  648 Aug 31 15:12 _c.fdt
-rw-r--r-- 1 root root   20 Aug 31 15:12 _c.fdx
-rw-r--r-- 1 root root  111 Aug 31 15:12 _c.fnm
-rw-r--r-- 1 root root  244 Aug 31 15:12 _c.frq
-rw-r--r-- 1 root root   10 Aug 31 15:12 _c.nrm
-rw-r--r-- 1 root root  366 Aug 31 15:12 _c.prx
-rw-r--r-- 1 root root   51 Aug 31 15:12 _c.tii
-rw-r--r-- 1 root root 1504 Aug 31 15:12 _c.tis
-rw-r--r-- 1 root root    0 Aug 31 15:12 lucene-85d8fe7216d62d59a3790bf2bb8c5e2b-n-write.lock
-rw-r--r-- 1 root root  700 Aug 31 15:12 segments_d

/index/slave/solr/data/spellchecker:
total 8
-rw-r--r-- 1 root root 20 Aug 25 23:37 segments.gen
-rw-r--r-- 1 root root 32 Aug 25 23:37 segments_1
{code}



> Replication causes master to fail to delete old index files
> -----------------------------------------------------------
>
>                 Key: SOLR-1383
>                 URL: https://issues.apache.org/jira/browse/SOLR-1383
>             Project: Solr
>          Issue Type: Bug
>          Components: replication (java)
>         Environment: Linux CentOS - latest Solr 1.4 trunk - Java 1.6
>            Reporter: Lance Norskog
>             Fix For: 1.4
>
>
> I have developed a way to make replication leave old index files in the master's data/index
directory. It is timing-dependent. A sequence of commands runs correctly or fails, depending
on the timing between the commands.
> Here is the test scenario:
> Start a master and slave version of the Solr distributed example. I used 8080 for the
slave. (See example/etc/jetty.xml)
> Be sure to start with empty solr/data/index files on both master and slave.
> Open the replication administration jsp on the slave ( http://localhost:8080/solr/admin/replication/index.jsp
)
> Disable polling.
> In a text window, go to the example/exampledocs directory and run this script
> {code}
> for x in *.xml
> do
> 	echo $x
> 	sh post.sh $x
> 	sleep 15
> 	curl "http://localhost:8080/solr/replication?command=fetchindex"
> done
> {code}
> This prints each example file, indexes it, and does a replication command. At the end
of this exercise, the master and slave solr/data/index files will be identical.
> Now, kill master & slave, remove the solr/index/data directories, and start over.
 This time, remove the sleep command from the script. In my environment, old Lucene index
files were left in the master's data/index. Here is what is left in the master data/index.

>  The segments_? files are random across runs, but the index files left over are consistent.
> Note (courtesy of the Linux 'ls -l /proc/PID/fd' command) that the old files are not
kept open by the master solr; they are merely left behind.
> In the master server:
> {code}
> % ls solr/data/index
> _0.fdt  _1.prx  _2.tvx  _4.nrm  _5.tii  _7.frq  _8.tvd  _a.tvx  _c.nrm
> _0.fdx  _1.tii  _3.fdt  _4.prx  _5.tis  _7.nrm  _8.tvf  _b.fdt  _c.prx
> _0.fnm  _1.tis  _3.fdx  _4.tii  _6.fdt  _7.prx  _8.tvx  _b.fdx  _c.tii
> _0.frq  _2.fdt  _3.fnm  _4.tis  _6.fdx  _7.tii  _a.fdt  _b.fnm  _c.tis
> _0.nrm  _2.fdx  _3.frq  _4.tvd  _6.fnm  _7.tis  _a.fdx  _b.frq  segments.gen
> _0.prx  _2.fnm  _3.nrm  _4.tvf  _6.frq  _8.fdt  _a.fnm  _b.nrm  segments_8
> _0.tii  _2.frq  _3.prx  _4.tvx  _6.nrm  _8.fdx  _a.frq  _b.prx  segments_9
> _0.tis  _2.nrm  _3.tii  _5.fdt  _6.prx  _8.fnm  _a.nrm  _b.tii  segments_a
> _1.fdt  _2.prx  _3.tis  _5.fdx  _6.tii  _8.frq  _a.prx  _b.tis  segments_b
> _1.fdx  _2.tii  _4.fdt  _5.fnm  _6.tis  _8.nrm  _a.tii  _c.fdt  segments_c
> _1.fnm  _2.tis  _4.fdx  _5.frq  _7.fdt  _8.prx  _a.tis  _c.fdx  segments_d
> _1.frq  _2.tvd  _4.fnm  _5.nrm  _7.fdx  _8.tii  _a.tvd  _c.fnm
> _1.nrm  _2.tvf  _4.frq  _5.prx  _7.fnm  _8.tis  _a.tvf  _c.frq
> {code}
> {code}
> % ls -l /proc/PID/fd
> lr-x------ 1 root root 64 Aug 25 22:52 137 -> /index/master/solr/data/index/_a.tis
> lr-x------ 1 root root 64 Aug 25 22:52 138 -> /index/master/solr/data/index/_a.frq
> lr-x------ 1 root root 64 Aug 25 22:52 139 -> /index/master/solr/data/index/_a.prx
> lr-x------ 1 root root 64 Aug 25 22:52 140 -> /index/master/solr/data/index/_a.fdt
> lr-x------ 1 root root 64 Aug 25 22:52 141 -> /index/master/solr/data/index/_a.fdx
> lr-x------ 1 root root 64 Aug 25 22:52 142 -> /index/master/solr/data/index/_a.tvx
> lr-x------ 1 root root 64 Aug 25 22:52 143 -> /index/master/solr/data/index/_a.tvd
> lr-x------ 1 root root 64 Aug 25 22:52 144 -> /index/master/solr/data/index/_a.tvf
> lr-x------ 1 root root 64 Aug 25 22:52 145 -> /index/master/solr/data/index/_a.nrm
> lr-x------ 1 root root 64 Aug 25 22:52 72 -> /index/master/solr/data/index/_b.tis
> lr-x------ 1 root root 64 Aug 25 22:52 73 -> /index/master/solr/data/index/_b.frq
> lr-x------ 1 root root 64 Aug 25 22:52 74 -> /index/master/solr/data/index/_b.prx
> lr-x------ 1 root root 64 Aug 25 22:52 76 -> /index/master/solr/data/index/_b.fdt
> lr-x------ 1 root root 64 Aug 25 22:52 78 -> /index/master/solr/data/index/_b.fdx
> lr-x------ 1 root root 64 Aug 25 22:52 79 -> /index/master/solr/data/index/_b.nrm
> lr-x------ 1 root root 64 Aug 25 22:52 80 -> /index/master/solr/data/index/_c.tis
> lr-x------ 1 root root 64 Aug 25 22:52 81 -> /index/master/solr/data/index/_c.frq
> lr-x------ 1 root root 64 Aug 25 22:52 82 -> /index/master/solr/data/index/_c.prx
> lr-x------ 1 root root 64 Aug 25 22:52 83 -> /index/master/solr/data/index/_c.fdt
> lr-x------ 1 root root 64 Aug 25 22:52 84 -> /index/master/solr/data/index/_c.fdx
> lr-x------ 1 root root 64 Aug 25 22:52 85 -> /index/master/solr/data/index/_c.nrm
> {code}
> In the slave server:
> {code}
> % ls solr/data/index
> _a.fdt  _a.tvd  _b.prx  _c.prx
> _a.fdx  _a.tvf  _b.tii  _c.tii
> _a.fnm  _a.tvx  _b.tis  _c.tis
> _a.frq  _b.fdt  _c.fdt  lucene-d81c111653e4c4883a6fbd7e2effd596-n-write.lock
> _a.nrm  _b.fdx  _c.fdx  segments.gen
> _a.prx  _b.fnm  _c.fnm  segments_d
> _a.tii  _b.frq  _c.frq
> _a.tis  _b.nrm  _c.nrm
> {code}
> {code}
> % ls -l /proc/PID/fd
> lr-x------ 1 root root 64 Aug 25 22:57 139 -> /index/slave/solr/data/index/_a.tis
> lr-x------ 1 root root 64 Aug 25 22:57 140 -> /index/slave/solr/data/index/_a.frq
> lr-x------ 1 root root 64 Aug 25 22:57 141 -> /index/slave/solr/data/index/_a.prx
> lr-x------ 1 root root 64 Aug 25 22:57 142 -> /index/slave/solr/data/index/_a.fdt
> lr-x------ 1 root root 64 Aug 25 22:57 143 -> /index/slave/solr/data/index/_a.fdx
> lr-x------ 1 root root 64 Aug 25 22:57 144 -> /index/slave/solr/data/index/_a.tvx
> lr-x------ 1 root root 64 Aug 25 22:57 145 -> /index/slave/solr/data/index/_a.tvd
> lr-x------ 1 root root 64 Aug 25 22:57 146 -> /index/slave/solr/data/index/_a.tvf
> lr-x------ 1 root root 64 Aug 25 22:57 147 -> /index/slave/solr/data/index/_a.nrm
> lr-x------ 1 root root 64 Aug 25 22:57 4 -> /index/slave/solr/data/index/_b.tis
> lr-x------ 1 root root 64 Aug 25 22:57 75 -> /index/slave/solr/data/index/_b.frq
> lr-x------ 1 root root 64 Aug 25 22:57 76 -> /index/slave/solr/data/index/_b.prx
> lr-x------ 1 root root 64 Aug 25 22:57 77 -> /index/slave/solr/data/index/_b.fdt
> lr-x------ 1 root root 64 Aug 25 22:57 78 -> /index/slave/solr/data/index/_b.fdx
> lr-x------ 1 root root 64 Aug 25 22:57 79 -> /index/slave/solr/data/index/_b.nrm
> lr-x------ 1 root root 64 Aug 25 22:57 80 -> /index/slave/solr/data/index/_c.tis
> lr-x------ 1 root root 64 Aug 25 22:57 81 -> /index/slave/solr/data/index/_c.frq
> lr-x------ 1 root root 64 Aug 25 22:57 82 -> /index/slave/solr/data/index/_c.prx
> lr-x------ 1 root root 64 Aug 25 22:57 83 -> /index/slave/solr/data/index/_c.fdt
> lr-x------ 1 root root 64 Aug 25 22:57 84 -> /index/slave/solr/data/index/_c.fdx
> lr-x------ 1 root root 64 Aug 25 22:57 85 -> /index/slave/solr/data/index/_c.nrm
> lrwx------ 1 root root 64 Aug 25 22:57 86 -> /index/slave/solr/data/index/lucene-d81c111653e4c4883a6fbd7e2effd596-n-write.lock
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message