lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Trcek <wzzelfz...@abas.de>
Subject Re: A lucene performance test
Date Mon, 13 Aug 2012 15:36:49 GMT
I can spend the program, it is attached.

----------------------------------------------------------------------

A short resumee:

* There must be 2 "caches" in the system: The performance degrades
  significantly beyond 1000 and again beyond 200000 documents
  in the index.

* JIT is very significant

* Index start up is very significant allthough the index is in the
  IO cache and the query is the most simple.

* The performance of index size 20000000 is strange.
  Worse performance than 50000000

----------------------------------------------------------------------
Environment:

lucene-3.5.0

$ cat /proc/sys/kernel/osrelease 
2.6.32-5-amd64

$ cat /proc/cpuinfo | tail ...
processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 42
model name	: Intel(R) Core(TM) i5-2500 CPU @ 3.30GHz
stepping	: 7
cpu MHz		: 1600.000
cache size	: 6144 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts
acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts
rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est
tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 x2apic popcnt aes xsave avx lahf_lm ida arat tpr_shadow
vnmi flexpriority ept vpid
bogomips	: 6584.99
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:
----------------------------------------------------------------------

No some results for writing:

$ java misc.LuceneKeyValuePerformance write
WRITING
#entries   seconds     entries/s
      1000       0,28       3607
      2000       0,26       7612
      5000       0,26      19176
     10000       0,20      50419
     20000       0,16     122044
     50000       0,26     191989
    100000       0,38     259987
    200000       0,72     276975
    500000       1,87     267547
   1000000       3,00     333436
   2000000       6,25     320167
   5000000      14,22     351645
  10000000      27,77     360082
  20000000      53,00     377385
  50000000     155,36     321839
 100000000     262,72     380636

# Now writing indexes in the opposite order.
# That show JIT startup

$ java misc.LuceneKeyValuePerformance writereverse
WRITING
#entries   seconds     entries/s
  20000000      56,39     354686
  10000000      26,90     371752
   5000000      13,60     367781
   2000000       6,17     324106
   1000000       2,97     336962
    500000       1,58     315545
    200000       0,59     336558
    100000       0,32     308275
     50000       0,25     198007
     20000       0,14     147298
     10000       0,17      57645
      5000       0,10      50244
      2000       0,12      16364
      1000       0,12       8587

Stefan

Mime
View raw message