lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <hossman_luc...@fucit.org>
Subject Re: 500 millions document for loop.
Date Sun, 15 Nov 2015 21:42:39 GMT

: 			public void collect(int docID) throws IOException {
: 				Document doc = indexSearcher.doc(docID, loadFields);
: 				found.found(doc);
: 			}

Based on your description of the calculation you are doing on all of these 
docs, you will probably find using DocValues on the "to" field and using 
that in your calculations will be a lot faster then dealing with the 
StoredFields...

: >>>>>> We have ~10 indexes for 500M documents, each document
: >>>>>> has «archive date», and «to» address, one of our task is
: >>>>>> calculate statistics of «to» for last year. Right now we are
: >>>>>> using search archive_date:(current_date - 1 year) and paginate


-Hoss
http://www.lucidworks.com/

Mime
View raw message