lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sangeetha.subramanian@gtnexus.com" <sangeetha.subraman...@gtnexus.com>
Subject Commit after every document - alternate approach
Date Wed, 02 Mar 2016 08:06:40 GMT
Hi All,

I am trying to understand on how we can have commit issued to solr while indexing documents.
Around 200K to 300K document/per hour with an avg size of 10 KB size each will be getting
into SOLR . JAVA code fetches the document from MQ and streamlines it to SOLR. The problem
is the client code issues hard-commit after each document which is sent to SOLR for indexing
and it waits for the response from SOLR to get assurance whether the document got indexed
successfully. Only if it gets a OK status from SOLR the document is cleared out from SOLR.

As far as I understand doing a commit after each document is an expensive operation. But we
need to make sure that all the documents which are put into MQ gets indexed in SOLR. Is there
any other way of getting this done ? Please let me know.
If we do a batch indexing, is there any chances we can identify if some documents is missed
from indexing ?

Thanks
Sangeetha

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message