lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Lea <ian....@gmail.com>
Subject Re: Using lucene in NFS
Date Fri, 30 Apr 2010 15:01:17 GMT
The suggestion was that your single indexing job should update a local
copy of the index and copy that to NFS for searching by other nodes.
That should work.

As for updating, you could index new reports into a new lucene index
and then merge that into the existing index
(IndexWriter.addIndexes()).  Or you could simple add them directly to
the existing index, making sure that you open it with create = false.


--
Ian.


On Fri, Apr 30, 2010 at 1:01 PM, Vijay Veeraraghavan
<vijay.raghavan08@gmail.com> wrote:
> hi Ian,
> Thanks for your reply. I am using the Lucene core 3.0 version. The
> index created will be accessed by the web application. The web
> application contains 4 nodes, clustered. What if all the nodes access
> the index. I think no any prob may raise.
> If i have a local index then what about it in the clustered environment?
> The NFS given is working good for saving the report, but only while i
> start the indexing the app freezes. it creates a lock file, i cant see
> whats happening inside as i cant see my console outputs too.
> And i have once more question. How do u update the index. As my
> scheduler keeps producing more reports to the file server, the index
> need to be updated then and there. how do i update the new report
> files to the index than doing again all the files. Or can i create a
> local index of the newly created reports and join this with the old
> index? any other professional solution?
>
> Thanks
> Vijay
>
> On 4/30/10, Ian Lea <ian.lea@gmail.com> wrote:
>> You don't say what version of lucene you are using, but in recent
>> versions you may need to use SimpleFSLockFactory rather than the
>> default, NativeFSLockFactory.  See the javadocs.  Lucene in general
>> does work on NFS but there can be problems, particularly with
>> concurrent access from multiple servers.
>>
>> But your problem sounds different.  Does the NFS mount work properly
>> for non-lucene operations?  Or maybe you could maintain a copy of the
>> index on local disks and just copy it to NFS for access from other
>> servers.  Lucene locking should take care of locking the index when
>> updates are running.
>>
>>
>> --
>> Ian.
>>
>>
>> On Fri, Apr 30, 2010 at 5:48 AM, Vijay Veeraraghavan
>> <vijay.raghavan08@gmail.com> wrote:
>>> dear all,
>>>
>>> I have a problem using lucene in NFS. A scheduler runs periodically
>>> generating reports in pdf format and saves it to a file server. The
>>> drive of the file server is mounted  to the scheduler server (NFS).
>>> After generating reports finally the scheduler indexes the names of
>>> the report and its path. This index is also stored to the same context
>>> folder the reports are created, ie., mounted NFS folder. When i create
>>> index in the local system (JFS2) its as fast as possible. (the local
>>> system is AIX uses jfs2.) But in the NFS folder it keep doing and
>>> doing... i didnt exactly know what happened inside. little later the
>>> application freezed.I saw some lock files created inside the folder
>>> but nothing like index files that usually created by the lucene. from
>>> the web page i search this index in nfs for the report and read it. My
>>> question is how to use lucene in NFS. anything special we need to do?
>>> any reference codes? docs? How do i lock the index when my scheduler
>>> is updating the index?
>>>
>>> P.S: Please add CC to vijayv@kggroup.com
>>>
>>> Thanks
>>> Vijay Veeraraghavan
>>>
>>>
>>>
>>> --
>>> Thanks & Regards
>>> Vijay Veeraraghavan
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
>
> --
> Thanks & Regards
> Vijay Veeraraghavan
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message