lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Della Bitta <michael.della.bi...@appinions.com>
Subject Re: solr problem after indexing, shutdown and startup
Date Wed, 21 May 2014 14:59:36 GMT
Two possibly unrelated things:

1. Don't commit until the end.

2. Consider not optimizing at all.

You might want to look at your autocommit settings in your solrconfig.xml.
You probably want soft commits set at something north of 10 seconds, and
hard commits set to openSearcher=false with a maxTime somewhat larger than
your soft commit setting, somewhere in the low minutes range.


Michael Della Bitta

Applications Developer

o: +1 646 532 3062

appinions inc.

“The Science of Influence Marketing”

18 East 41st Street

New York, NY 10017

t: @appinions <https://twitter.com/Appinions> | g+:
plus.google.com/appinions<https://plus.google.com/u/0/b/112002776285509593336/112002776285509593336/posts>
w: appinions.com <http://www.appinions.com/>


On Wed, May 21, 2014 at 9:51 AM, Cam Bazz <cambazz@gmail.com> wrote:

> Hello,
>
> I am indexing some 20 million documents in a solr instance.
>
> After i do the indexing, and shut the instance down and back on, it will
> not respond to queries,
> and it will not show collection stats.
>
> I am attaching the solr log to this email.
>
> The way that I do indexing is:
>
>         // document list
>         Collection<SolrInputDocument> docs = new ArrayList<>();
>
>         // process files
>         for(;;) {
>
>                // create document and add it to docs list
>                 if(linenum%(256*128)==0) {
>                    // every once in a while add them to server
>                     try {
>                         server.add(docs);
>                         server.commit(true, false, false);
>                         docs.clear();
>                     } catch (SolrServerException ex) {
>
> Logger.getLogger(Indexer.class.getName()).log(Level.SEVERE, null, ex);
>                     }
>                 }
>       }
>
>             // and at the end i optimize
>             try {
>                 server.optimize(true, false);
>             } catch (SolrServerException ex) {
>
> Logger.getLogger(Indexer.class.getName()).log(Level.SEVERE, null, ex);
>             }
>
>
> I am suspecting this has to do something with background merges, etc.
>
> Any ideas/help/recomendations on this problem is greatly appreciated. I am
> using solr4.8
>
> Best Regards,
> C.B.
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message