lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Umashanker, Srividhya" <srividhya.umashan...@hp.com>
Subject [Suggestions Required] 110 Concurrency users indexing on Lucene dont finish in 200 ms.
Date Tue, 11 Feb 2014 05:25:27 GMT
Group -



We have an Indexing and Searching Service (using Lucene 4.0) implemented over REST  as part
of our framework, which all the related modules will use to publish data  that make it available
for the UI.

Moreover, every rest call that our service receives has a proxy timeout limit of 200 ms.

Initially we started concurrency tests with Indexing.   Recently, we started doing some concurrency
tests on our service to handle  min 110 users, who will be publishing json data ranging from
10, 50 100 documents by one user. The document is not huge like what u are using for your
performance tests. Mostly we call updateDocument() as we do not know whether the document
is already indexed or not



110 users/threads with a ramp up time of 0 seconds seems to fail randomly because some of
the threads/REST call do not complete within 200 ms.



We tried the following.

1. maxBufferSize of 500 MB.

2. DWPT count to 10.

3. Tried updateDocument in separate thread, as said in one of your suggestions in the internet.
Doesn't seem to help.

4. TiredMergePolicy has been set. Unfortunately this does not help as the maxBufferSize is
not hit while we index, and merge does not happen yet.

5. We are yet to try with NMapDirectory.



Are there any other suggestions/experiences that you can share with us.  Are there any benchmark
tests that you have done on concurrency?



Please help us.



-Vidhya



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message