lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: Solr in NAS or Network Shared Drive
Date Wed, 24 May 2017 02:05:29 GMT
On 5/19/2017 8:33 AM, Ravi Kumar Taminidi wrote:
> Hello,  Scenario: Currently we have 2 Solr Servers running in 2 different servers (linux),
Is there any way can we make the Core to be located in NAS or Network shared Drive so both
the solrs using the same Index.
>
> Let me know if any performance issues, our size of Index is appx 1GB.

I think it's a very bad idea to try to share indexes between multiple
Solr instances.  You can override the locking and get it to work, and
you may be able to find advice on the Internet about how to do it.  I
can tell you that it's outside the design intent for both Lucene and
Solr.  Lucene works aggressively to *prevent* multiple processes from
sharing an index.

In general, network storage is not a good idea for Solr.  There's added
latency for accessing any data, and frequently the filesystem won't
support the kind of locking that Lucene wants to use, but the biggest
potential problem is disk caching.  Solr/Lucene is absolutely reliant on
disk caching in the SOlr server's local memory for good performance.  If
the network filesystem cannot be cached by the client that has mounted
the storage, which I believe is the case for most network filesystem
types, then you're reliant on disk caching in the network server(s). 
For VERY large indexes, which is really the only viable use case I can
imagine for network storage, it is highly unlikely that the network
server(s) will have enough memory to effectively cache the data.

Solr has explicit support for HDFS storage, but as I understand it, HDFS
includes the ability for a client to allocate memory that gets used
exclusively for caching on the client side, which allows HDFS to
function like a local filesystem in ways that I don't think NFS can. 
Getting back to my advice about not sharing indexes -- even with
SolrCloud on HDFS, multiple replicas generally do NOT share an index.

A 1GB index is very small, so there's no good reason I can think of to
involve network storage.  I would strongly recommend local storage, and
you should abandon any attempt to share the same index data between more
than one Solr instance.

Thanks,
Shawn


Mime
View raw message