lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Noble Paul നോബിള്‍ नोब्ळ्" <noble.p...@gmail.com>
Subject Re: "Unable to move index file" error during replication
Date Wed, 24 Dec 2008 10:54:24 GMT
James thanks .

If this is true the place to fix this is in
ReplicationHandler#getFileList(). patch is attached.


On Wed, Dec 24, 2008 at 4:04 PM, James Grant <James.Grant@semantico.com> wrote:
> I had the same problem. It turned out that the list of files from the master
> included duplicates. When the slave completes the download and tries to move
> the files into the index it comes across a file that does not exist because
> it has already been moved so it backs out the whole operation.
>
> My solution for now was to patch the copyindexFiles method of
> org.apache.solr.handler.SnapPuller so that it normalises the list before
> moving the files. This isn't the best solution since it will still download
> the file twice but it was the easiest and smallest change to make. The patch
> is below
>
> Regards
>
> James
>
> --- src/java/org/apache/solr/handler/SnapPuller.java    (revision 727347)
> +++ src/java/org/apache/solr/handler/SnapPuller.java    (working copy)
> @@ -470,7 +470,7 @@
>   */
>  private boolean copyIndexFiles(File snapDir, File indexDir) {
>    String segmentsFile = null;
> -    List<String> copiedfiles = new ArrayList<String>();
> +    Set<String> filesToCopy = new HashSet<String>();
>    for (Map<String, Object> f : filesDownloaded) {
>      String fname = (String) f.get(NAME);
>      // the segments file must be copied last
> @@ -482,6 +482,10 @@
>        segmentsFile = fname;
>        continue;
>      }
> +      filesToCopy.add(fname);
> +    }
> +    List<String> copiedfiles = new ArrayList<String>();
> +    for (String fname: filesToCopy) {
>      if (!copyAFile(snapDir, indexDir, fname, copiedfiles)) return false;
>      copiedfiles.add(fname);
>    }
>
>
> Jaco wrote:
>>
>> Hello,
>>
>> While testing out the new replication features, I'm running into some
>> strange problem. On the slave, I keep getting an error like this after all
>> files have been copied from the master to the temporary index.xxxxxxxxx
>> directory:
>>
>> SEVERE: Unable to move index file from:
>> D:\Data\solr\Slave\data\index.20081224110855\_21e.tvx to:
>> D:\Data\Solr\Slave\data\index\_21e.tvx
>>
>> The replication then stops, index remains in original state, so the
>> updates
>> are not available at the slave.
>>
>> This is my replication config at the master:
>>
>>    <requestHandler name="/replication" class="solr.ReplicationHandler" >
>>        <lst name="master">
>>            <!--Replicate on 'optimize' it can also be  'commit' -->
>>            <str name="replicateAfter">commit</str>
>>            <str name="confFiles">schema.xml</str>
>>        </lst>
>>    </requestHandler>
>>
>> This is the replication config at the slave:
>>
>>    <requestHandler name="/replication" class="solr.ReplicationHandler" >
>>        <lst name="slave">
>>            <str name="masterUrl">
>> http://hostnamemaster:8080/solr/Master/replication</str>
>>            <str name="pollInterval">00:10:00</str>
>>            <str name="zip">true</str>
>>        </lst>
>>    </requestHandler>
>>
>> I'm running a Solr nightly build of 21.12.2008 in Tomcat 6 on Windows
>> 2003.
>> Initially I thought there was some problem with disk space, but this is
>> not
>> the case. Replication did run fine for intial version of index, but after
>> that at some point it didn't work anymore. Any ideas what could be wrong
>> here?
>>
>> Thanks very much in advance, bye,
>>
>> Jaco.
>>
>>
>
>



-- 
--Noble Paul

Mime
View raw message