lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tomás Fernández Löbbe <tomasflo...@gmail.com>
Subject Re: Simple Slave Replication Question
Date Fri, 23 Mar 2012 15:10:27 GMT
Also, what happens if, instead of adding the 40K docs you add just one and
commit?

2012/3/23 Tomás Fernández Löbbe <tomasflobbe@gmail.com>

> Have you changed the mergeFactor or are you using 10 as in the example
> solrconfig?
>
> What do you see in the slave's log during replication? Do you see any line
> like "Skipping download for..."?
>
>
> On Fri, Mar 23, 2012 at 11:57 AM, Ben McCarthy <
> Ben.McCarthy@tradermedia.co.uk> wrote:
>
>> I just have a index directory.
>>
>> I push the documents through with a change to a field.  Im using SOLRJ to
>> do this.  Im using the guide from the wiki to setup the replication.  When
>> the feed of updates to the master finishes I call a commit again using
>> SOLRJ.  I then have a poll period of 5 minutes from the slave.  When it
>> kicks in I see a new version of the index and then it copys the full 5gb
>> index.
>>
>> Thanks
>> Ben
>>
>> -----Original Message-----
>> From: Tomás Fernández Löbbe [mailto:tomasflobbe@gmail.com]
>> Sent: 23 March 2012 14:29
>> To: solr-user@lucene.apache.org
>> Subject: Re: Simple Slave Replication Question
>>
>> Hi Ben, only new segments are replicated from master to slave. In a
>> situation where all the segments are new, this will cause the index to be
>> fully replicated, but this rarely happen with incremental updates. It can
>> also happen if the slave Solr assumes it has an "invalid" index.
>> Are you committing or optimizing on the slaves? After replication, the
>> index directory on the slaves is called "index" or "index.<timestamp>"?
>>
>> Tomás
>>
>> On Fri, Mar 23, 2012 at 11:18 AM, Ben McCarthy <
>> Ben.McCarthy@tradermedia.co.uk> wrote:
>>
>> > So do you just simpy address this with big nic and network pipes.
>> >
>> > -----Original Message-----
>> > From: Martin Koch [mailto:mak@issuu.com]
>> > Sent: 23 March 2012 14:07
>> > To: solr-user@lucene.apache.org
>> > Subject: Re: Simple Slave Replication Question
>> >
>> > I guess this would depend on network bandwidth, but we move around
>> > 150G/hour when hooking up a new slave to the master.
>> >
>> > /Martin
>> >
>> > On Fri, Mar 23, 2012 at 12:33 PM, Ben McCarthy <
>> > Ben.McCarthy@tradermedia.co.uk> wrote:
>> >
>> > > Hello,
>> > >
>> > > Im looking at the replication from a master to a number of slaves.
>> > > I have configured it and it appears to be working.  When updating
>> > > 40K records on the master is it standard to always copy over the
>> > > full index, currently 5gb in size.  If this is standard what do
>> > > people do who have massive 200gb indexs, does it not take a while to
>> > > bring the
>> > slaves inline with the master?
>> > >
>> > > Thanks
>> > > Ben
>> > >
>> > > ________________________________________
>> > >
>> > >
>> > > This e-mail is sent on behalf of Trader Media Group Limited,
>> > > Registered
>> > > Office: Auto Trader House, Cutbush Park Industrial Estate, Danehill,
>> > > Lower Earley, Reading, Berkshire, RG6 4UT(Registered in England No.
>> > 4768833).
>> > > This email and any files transmitted with it are confidential and
>> > > may be legally privileged, and intended solely for the use of the
>> > > individual or entity to whom they are addressed. If you have
>> > > received this email in error please notify the sender. This email
>> > > message has been swept for the presence of computer viruses.
>> > >
>> > >
>> >
>> > ________________________________________
>> >
>> >
>> > This e-mail is sent on behalf of Trader Media Group Limited,
>> > Registered
>> > Office: Auto Trader House, Cutbush Park Industrial Estate, Danehill,
>> > Lower Earley, Reading, Berkshire, RG6 4UT(Registered in England No.
>> 4768833).
>> > This email and any files transmitted with it are confidential and may
>> > be legally privileged, and intended solely for the use of the
>> > individual or entity to whom they are addressed. If you have received
>> > this email in error please notify the sender. This email message has
>> > been swept for the presence of computer viruses.
>> >
>> >
>>
>> ________________________________________
>>
>>
>> This e-mail is sent on behalf of Trader Media Group Limited, Registered
>> Office: Auto Trader House, Cutbush Park Industrial Estate, Danehill, Lower
>> Earley, Reading, Berkshire, RG6 4UT(Registered in England No. 4768833).
>> This email and any files transmitted with it are confidential and may be
>> legally privileged, and intended solely for the use of the individual or
>> entity to whom they are addressed. If you have received this email in error
>> please notify the sender. This email message has been swept for the
>> presence of computer viruses.
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message