lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edwin Khodabakchian (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SOLR-2329) old index files not deleted on slave
Date Wed, 19 Sep 2012 03:17:07 GMT

     [ https://issues.apache.org/jira/browse/SOLR-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Edwin Khodabakchian updated SOLR-2329:
--------------------------------------

    Attachment:     (was: solrconfig_slave.xml)
    
> old index files not deleted on slave
> ------------------------------------
>
>                 Key: SOLR-2329
>                 URL: https://issues.apache.org/jira/browse/SOLR-2329
>             Project: Solr
>          Issue Type: Bug
>          Components: replication (java)
>    Affects Versions: 4.0-ALPHA
>         Environment: centos 5.5
> ext3 file system
>            Reporter: Edwin Khodabakchian
>         Attachments: solrconfig.xml
>
>
> I have set up index replication (triggered on optimize). The problem I
> am having is the old index files are not being deleted on the slave.
> After each replication, I can see the old files still hanging around
> as well as the files that have just been pulled. This causes the data
> directory size to increase by the index size every replication until
> the disk fills up.
> I am running 4.0 rev 993367 with patch SOLR-1316. Otherwise, the setup
> is pretty vanilla. I can reproduce this on multiple slaves.
> Checking the logs, I see the following error:
> SEVERE: SnapPull failed
> org.apache.solr.common.SolrException: Index fetch failed :
>        at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:329)
>        at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:265)
>        at org.apache.solr.handler.SnapPuller$1.run(SnapPuller.java:159)
>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>        at java.lang.Thread.run(Thread.java:619)
> Caused by: org.apache.lucene.store.LockObtainFailedException: Lock
> obtain timed out:
> NativeFSLock@/var/solrhome/data/index/lucene-cdaa80c0fefe1a7dfc7aab89298c614c-write.lock
>        at org.apache.lucene.store.Lock.obtain(Lock.java:84)
>        at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1065)
>        at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:954)
>        at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:192)
>        at org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:99)
>        at org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:173)
>        at org.apache.solr.update.DirectUpdateHandler2.forceOpenWriter(DirectUpdateHandler2.java:376)
>        at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:471)
>        at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:319)
>        ... 11 more
> lsof reveals that the file is still opened from the java process.
> Contents of the index data dir:
> master:
> -rw-rw-r-- 1 feeddo feeddo  191 Dec 14 01:06 _1lg.fnm
> -rw-rw-r-- 1 feeddo feeddo  26M Dec 14 01:07 _1lg.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 14 01:07 _1lg.fdt
> -rw-rw-r-- 1 feeddo feeddo 474M Dec 14 01:12 _1lg.tis
> -rw-rw-r-- 1 feeddo feeddo  15M Dec 14 01:12 _1lg.tii
> -rw-rw-r-- 1 feeddo feeddo 144M Dec 14 01:12 _1lg.prx
> -rw-rw-r-- 1 feeddo feeddo 277M Dec 14 01:12 _1lg.frq
> -rw-rw-r-- 1 feeddo feeddo  311 Dec 14 01:12 segments_1ji
> -rw-rw-r-- 1 feeddo feeddo  23M Dec 14 01:12 _1lg.nrm
> -rw-rw-r-- 1 feeddo feeddo  191 Dec 18 01:11 _24e.fnm
> -rw-rw-r-- 1 feeddo feeddo  26M Dec 18 01:12 _24e.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 18 01:12 _24e.fdt
> -rw-rw-r-- 1 feeddo feeddo 483M Dec 18 01:23 _24e.tis
> -rw-rw-r-- 1 feeddo feeddo  15M Dec 18 01:23 _24e.tii
> -rw-rw-r-- 1 feeddo feeddo 146M Dec 18 01:23 _24e.prx
> -rw-rw-r-- 1 feeddo feeddo 283M Dec 18 01:23 _24e.frq
> -rw-rw-r-- 1 feeddo feeddo  311 Dec 18 01:24 segments_1xz
> -rw-rw-r-- 1 feeddo feeddo  23M Dec 18 01:24 _24e.nrm
> -rw-rw-r-- 1 feeddo feeddo  191 Dec 18 13:15 _25z.fnm
> -rw-rw-r-- 1 feeddo feeddo  26M Dec 18 13:16 _25z.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 18 13:16 _25z.fdt
> -rw-rw-r-- 1 feeddo feeddo 484M Dec 18 13:35 _25z.tis
> -rw-rw-r-- 1 feeddo feeddo  15M Dec 18 13:35 _25z.tii
> -rw-rw-r-- 1 feeddo feeddo 146M Dec 18 13:35 _25z.prx
> -rw-rw-r-- 1 feeddo feeddo 284M Dec 18 13:35 _25z.frq
> -rw-rw-r-- 1 feeddo feeddo   20 Dec 18 13:35 segments.gen
> -rw-rw-r-- 1 feeddo feeddo  311 Dec 18 13:35 segments_1y1
> -rw-rw-r-- 1 feeddo feeddo  23M Dec 18 13:35 _25z.nrm
> slave:
> -rw-rw-r-- 1 feeddo feeddo   20 Dec 13 17:54 segments.gen
> -rw-rw-r-- 1 feeddo feeddo  191 Dec 15 01:07 _1mk.fnm
> -rw-rw-r-- 1 feeddo feeddo  26M Dec 15 01:08 _1mk.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 15 01:08 _1mk.fdt
> -rw-rw-r-- 1 feeddo feeddo 476M Dec 15 01:18 _1mk.tis
> -rw-rw-r-- 1 feeddo feeddo  15M Dec 15 01:18 _1mk.tii
> -rw-rw-r-- 1 feeddo feeddo 144M Dec 15 01:18 _1mk.prx
> -rw-rw-r-- 1 feeddo feeddo 278M Dec 15 01:18 _1mk.frq
> -rw-rw-r-- 1 feeddo feeddo  312 Dec 15 01:18 segments_1kj
> -rw-rw-r-- 1 feeddo feeddo  23M Dec 15 01:18 _1mk.nrm
> -rw-rw-r-- 1 feeddo feeddo    0 Dec 15 01:19
> lucene-cdaa80c0fefe1a7dfc7aab89298c614c-write.lock
> -rw-rw-r-- 1 feeddo feeddo  191 Dec 15 13:14 _1qu.fnm
> -rw-rw-r-- 1 feeddo feeddo  26M Dec 15 13:16 _1qu.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 15 13:16 _1qu.fdt
> -rw-rw-r-- 1 feeddo feeddo 477M Dec 15 13:28 _1qu.tis
> -rw-rw-r-- 1 feeddo feeddo  15M Dec 15 13:28 _1qu.tii
> -rw-rw-r-- 1 feeddo feeddo 144M Dec 15 13:28 _1qu.prx
> -rw-rw-r-- 1 feeddo feeddo 278M Dec 15 13:28 _1qu.frq
> -rw-rw-r-- 1 feeddo feeddo  311 Dec 15 13:28 segments_1oe
> -rw-rw-r-- 1 feeddo feeddo  23M Dec 15 13:28 _1qu.nrm
> -rw-rw-r-- 1 feeddo feeddo  191 Dec 17 01:12 _222.fnm
> -rw-rw-r-- 1 feeddo feeddo  26M Dec 17 01:15 _222.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 17 01:15 _222.fdt
> -rw-rw-r-- 1 feeddo feeddo 481M Dec 17 01:36 _222.tis
> -rw-rw-r-- 1 feeddo feeddo  15M Dec 17 01:36 _222.tii
> -rw-rw-r-- 1 feeddo feeddo 145M Dec 17 01:36 _222.prx
> -rw-rw-r-- 1 feeddo feeddo 281M Dec 17 01:36 _222.frq
> -rw-rw-r-- 1 feeddo feeddo  23M Dec 17 01:36 _222.nrm
> -rw-rw-r-- 1 feeddo feeddo  311 Dec 17 01:36 segments_1xv
> -rw-rw-r-- 1 feeddo feeddo  191 Dec 17 13:10 _233.fnm
> -rw-rw-r-- 1 feeddo feeddo  26M Dec 17 13:13 _233.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 17 13:13 _233.fdt
> -rw-rw-r-- 1 feeddo feeddo 482M Dec 17 13:31 _233.tis
> -rw-rw-r-- 1 feeddo feeddo  15M Dec 17 13:31 _233.tii
> -rw-rw-r-- 1 feeddo feeddo 146M Dec 17 13:31 _233.prx
> -rw-rw-r-- 1 feeddo feeddo 282M Dec 17 13:31 _233.frq
> -rw-rw-r-- 1 feeddo feeddo  311 Dec 17 13:31 segments_1xx
> -rw-rw-r-- 1 feeddo feeddo  23M Dec 17 13:31 _233.nrm
> -rw-rw-r-- 1 feeddo feeddo  191 Dec 18 01:11 _24e.fnm
> -rw-rw-r-- 1 feeddo feeddo  26M Dec 18 01:12 _24e.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 18 01:12 _24e.fdt
> -rw-rw-r-- 1 feeddo feeddo 483M Dec 18 01:23 _24e.tis
> -rw-rw-r-- 1 feeddo feeddo  15M Dec 18 01:23 _24e.tii
> -rw-rw-r-- 1 feeddo feeddo 146M Dec 18 01:23 _24e.prx
> -rw-rw-r-- 1 feeddo feeddo 283M Dec 18 01:23 _24e.frq
> -rw-rw-r-- 1 feeddo feeddo  311 Dec 18 01:24 segments_1xz
> -rw-rw-r-- 1 feeddo feeddo  23M Dec 18 01:24 _24e.nrm
> -rw-rw-r-- 1 feeddo feeddo  191 Dec 18 13:15 _25z.fnm
> -rw-rw-r-- 1 feeddo feeddo  26M Dec 18 13:16 _25z.fdx
> -rw-rw-r-- 1 feeddo feeddo 1.9G Dec 18 13:16 _25z.fdt
> -rw-rw-r-- 1 feeddo feeddo 484M Dec 18 13:35 _25z.tis
> -rw-rw-r-- 1 feeddo feeddo  15M Dec 18 13:35 _25z.tii
> -rw-rw-r-- 1 feeddo feeddo 146M Dec 18 13:35 _25z.prx
> -rw-rw-r-- 1 feeddo feeddo 284M Dec 18 13:35 _25z.frq
> -rw-rw-r-- 1 feeddo feeddo  311 Dec 18 13:35 segments_1y1
> -rw-rw-r-- 1 feeddo feeddo  23M Dec 18 13:35 _25z.nrm

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message