lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Miller (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-1781) Replication index directories not always cleaned up
Date Tue, 24 Jul 2012 17:21:34 GMT

    [ https://issues.apache.org/jira/browse/SOLR-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421557#comment-13421557
] 

Mark Miller commented on SOLR-1781:
-----------------------------------

Odd - in the first case, you are sure the indexes were there over time? For brief periods,
more than once can exist...it should just end up being cleaned up when no longer in use.

I can try and dig in some more, but I'll have to think a little - I don't really know where
to start.

My test for this issue is a test that runs a lot of instances and randomly starts and stop
them. I then monitor the index directories for these 6-12 instances - I run these tests for
a long time and monitor that each keeps ending up with one index dir. At some points, there
are two indexes - but it always drops to one shortly later.

So there still may be some hole, but I don't know where or how.

If you look in the logs, perhaps you will see a bunch of "Unable to delete directory : " entries?
It might be that it's trying but cannot. It might make sense to start using deleteOnExit as
a last resort if delete fails - I just looked and the del call in SnapPuller does not do this.
                
> Replication index directories not always cleaned up
> ---------------------------------------------------
>
>                 Key: SOLR-1781
>                 URL: https://issues.apache.org/jira/browse/SOLR-1781
>             Project: Solr
>          Issue Type: Bug
>          Components: replication (java), SolrCloud
>    Affects Versions: 1.4
>         Environment: Windows Server 2003 R2, Java 6b18
>            Reporter: Terje Sten Bjerkseth
>            Assignee: Mark Miller
>             Fix For: 4.0, 5.0
>
>         Attachments: 0001-Replication-does-not-always-clean-up-old-directories.patch,
SOLR-1781.patch, SOLR-1781.patch
>
>
> We had the same problem as someone described in http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201001.mbox/%3c222A518D-DDF5-4FC8-A02A-74D4F232B339@snooth.com%3e.
A partial copy of that message:
> We're using the new replication and it's working pretty well. There's  
> one detail I'd like to get some more information about.
> As the replication works, it creates versions of the index in the data  
> directory. Originally we had index/, but now there are dated versions  
> such as index.20100127044500/, which are the replicated versions.
> Each copy is sized in the vicinity of 65G. With our current hard drive  
> it's fine to have two around, but 3 gets a little dicey. Sometimes  
> we're finding that the replication doesn't always clean up after  
> itself. I would like to understand this better, or to not have this  
> happen. It could be a configuration issue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message