lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Hill <jayallenh...@gmail.com>
Subject Batching requests using SolrCell with SolrJ
Date Sat, 19 Sep 2009 17:22:54 GMT
When working with SolrJ I have typically batched a Collection of
SolrInputDocument objects before sending them to the Solr server. I'm
working with the latest nightly build and using the ExtractingRequestHandler
to index documents, and everything is working fine. Except I haven't been
able to figure out how to batch documents when also including literals.
Here's what I've got:

//Looping over a List of Files
          ContentStreamUpdateRequest req = new
ContentStreamUpdateRequest("/update/extract");
          req.addFile(fileToIndex);
          req.setParam("literal.id", fileToIndex.getCanonicalPath());

          try {
            getSolrServer().request(req);
          } catch (SolrServerException e) {
            e.printStackTrace();
          }

Which works great, except that each document processed in the loop is
sending a separate request. Previously I built a collection of SolrInput
docs and had SolrJ send them in batches of 100 or whatever.

It seems like I could batch documents by continuing to add them to the
request (req.addFile(eachFileUpToACount)), but the literals seem to present
a problem. By sending one at a time the contents and the literals all wind
up in the same document. But in a batch there will just be an array of
params for literal.id (in this example) not matched to the contents.

Can anyone provide a code snippet of how to do this? Or is there no other
approach than sending a request for each document.

Thanks,
-Jay
http://www.lucidimagination.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message