lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis.gospodne...@gmail.com>
Subject Re: Storing Solr Index on NFS
Date Tue, 16 Apr 2013 22:52:07 GMT
No.  This is about 1 Solr server. You don't need to do anything with caches.

Otis
--
Solr & ElasticSearch Support
http://sematext.com/





On Tue, Apr 16, 2013 at 6:50 PM, Furkan KAMACI <furkankamaci@gmail.com> wrote:
> I don't want to bother but I try to understand that part:
>
> "When yo perform a commit in solr you have (for an instant) two versions of
> the index. The commit produces new segments (with new documents, new
> deletions, etc). After creating these new segments a new index searcher is
> created and its caches begin to autowarm. At this point the old index
> searcher that you were using is still active receiving requests. After the
> new index searcher finishes loading and autowarming the old searcher is
> discarded."
>
> So does it mean that when I have multiple Solr servers and a shared index,
> I should synchronize the caches at that different machines RAMs?
>
> 2013/4/17 Otis Gospodnetic <otis.gospodnetic@gmail.com>
>
>> Yesterday, we spent 1 hour with a client looking at their cluster's
>> performance metrics SPM, their indexing logs, etc. trying to figure
>> out why some indexing was slower than it should have been.  We traced
>> issues to network hickups, to VMs that would move from host to host,
>> etc.  Really fancy and powerful system in terms of hardware resources,
>> but in the end a bit too far from just locally attached HDD or SDD
>> that would not have issues like the ones we found.  I'd stay away from
>> NFS for the same reason - it's another moving part on the other side
>> of the network.
>>
>> Otis
>> --
>> Solr & ElasticSearch Support
>> http://sematext.com/
>>
>>
>>
>>
>>
>> On Tue, Apr 16, 2013 at 7:15 AM, Furkan KAMACI <furkankamaci@gmail.com>
>> wrote:
>> > Hi Walter;
>> >
>> > You said: "It is not safe to share Solr index files between two Solr
>> > servers". Why do you think like that?
>> >
>> >
>> > 2013/4/16 Tim Vaillancourt <tim@elementspace.com>
>> >
>> >> If centralization of storage is your goal by choosing NFS, iSCSI works
>> >> reasonably well with SOLR indexes, although good local-storage will
>> always
>> >> be the overall winner.
>> >>
>> >> I noticed a near 5% degredation in overall search performance (casual
>> >> testing, nothing scientific) when moving a 40-50GB indexes to iSCSI
>> (10GBe
>> >> network) from a 4x7200rpm RAID 10 local SATA disk setup.
>> >>
>> >> Tim
>> >>
>> >>
>> >> On 15/04/13 09:59 AM, Walter Underwood wrote:
>> >>
>> >>> Solr 4.2 does have field compression which makes smaller indexes. That
>> >>> will reduce the amount of network traffic. That probably does not help
>> >>> much, because I think the latency of NFS is what causes problems.
>> >>>
>> >>> wunder
>> >>>
>> >>> On Apr 15, 2013, at 9:52 AM, Ali, Saqib wrote:
>> >>>
>> >>>  Hello Walter,
>> >>>>
>> >>>> Thanks for the response. That has been my experience in the past
as
>> well.
>> >>>> But I was wondering if there new are things in Solr 4 and NFS 4.1
that
>> >>>> make
>> >>>> the storing of indexes on a NFS mount feasible.
>> >>>>
>> >>>> Thanks,
>> >>>> Saqib
>> >>>>
>> >>>>
>> >>>> On Mon, Apr 15, 2013 at 9:47 AM, Walter Underwood<wunder@wunderwood.
>> **
>> >>>> org <wunder@wunderwood.org>>wrote:
>> >>>>
>> >>>>  On Apr 15, 2013, at 9:40 AM, Ali, Saqib wrote:
>> >>>>>
>> >>>>>  Greetings,
>> >>>>>>
>> >>>>>> Are there any issues with storing Solr Indexes on a NFS
share? Also
>> any
>> >>>>>> recommendations for using NFS for Solr indexes?
>> >>>>>>
>> >>>>> I recommend that you do not put Solr indexes on NFS.
>> >>>>>
>> >>>>> It can be very slow, I measured indexing as 100X slower on NFS
a few
>> >>>>> years
>> >>>>> ago.
>> >>>>>
>> >>>>> It is not safe to share Solr index files between two Solr servers,
so
>> >>>>> there is no benefit to NFS.
>> >>>>>
>> >>>>> wunder
>> >>>>> --
>> >>>>> Walter Underwood
>> >>>>> wunder@wunderwood.org
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>  --
>> >>> Walter Underwood
>> >>> wunder@wunderwood.org
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>>

Mime
View raw message