lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <>
Subject Re: Benchmarking my indexer
Date Sun, 02 Nov 2008 12:11:20 GMT

On Nov 1, 2008, at 1:39 AM, Rafael Cunha de Almeida wrote:

> Hello,
> I did an indexer that parses some files and indexes them using  
> lucene. I
> want to benchmark the whole thing, so I'd like to count the tokens
> being indexed so I can calculate the average number of indexed tokens
> per second. Is there a way to count the number of tokens on a  
> document?

I think you would have to add a "CountingTokenFilter", that you write  
and manage as you add documents.  Or, you could just take the total #  
of tokens / by the number of docs and use the average.  That can be  
obtained w/o writing a new TokenFilter.

> While I'm at it, I will also need to calculate the amount of memory my
> java program used (peak, avg, etc), what java tool would you suggest  
> me
> to figure that out?

Would JConsole work: 
   help?  I'm not sure what people use here

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message