lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "hui" <...@triplehop.com>
Subject RE: Performance: compound vs. multi-file index, indexing and searching
Date Sat, 12 Jun 2004 06:30:28 GMT
Please see the test results for the 3 situations on Lucene 1.4rc3. 
Index speed from fast to slow
	Multiple files indexing with minMergeDoc=100 -- Multiple files
indexing with minMergeDoc=10 -- compound index
The search times are almost the same.

Regards,
hui



tranditional indexing with default minMergeDoc=10
1360615 total milliseconds index total docs:66100
1361646 total milliseconds index total docs:66100
1438868 total milliseconds index total docs:66100
1406569 total milliseconds index total docs:66100
1405461 total milliseconds index total docs:66100
1428664 total milliseconds index total docs:66100
1436430 total milliseconds index total docs:66100
1527652 total milliseconds index total docs:66100
1757083 total milliseconds index total docs:66100

compound indexing
1539246 total milliseconds index total docs:66100
1370881 total milliseconds index total docs:66100
1373428 total milliseconds index total docs:66100
1410663 total milliseconds index total docs:66100
1472197 total milliseconds index total docs:66100
1509933 total milliseconds index total docs:66100
1648204 total milliseconds index total docs:66100
1835086 total milliseconds index total docs:66100
1984420 total milliseconds index total docs:66100
2051743 total milliseconds index total docs:66100


tranditional indexing with default minMergeDoc=100
1543497 total milliseconds index total docs:66100
1170030 total milliseconds index total docs:66100
1174342 total milliseconds index total docs:66100
1185701 total milliseconds index total docs:66100
1205703 total milliseconds index total docs:66100
1210093 total milliseconds index total docs:66100
1228907 total milliseconds index total docs:66100
1323254 total milliseconds index total docs:66100
1578763 total milliseconds index total docs:66100
1593499 total milliseconds index total docs:66100


trandistional index search while indexing with default minMergeDoc=10
0 open index time:156
571 total within(ms) 110
602 total within(ms) 16
724 total within(ms) 47
449 total within(ms) 16
18 total within(ms) 0
1 open index time:63
4204 total within(ms) 0
5593 total within(ms) 0
6655 total within(ms) 16
3142 total within(ms) 15
164 total within(ms) 31
2 open index time:47
8243 total within(ms) 16
10920 total within(ms) 16
13025 total within(ms) 78
6138 total within(ms) 16
332 total within(ms) 16
3 open index time:62
12194 total within(ms) 16
16212 total within(ms) 16
19307 total within(ms) 31
9099 total within(ms) 15
485 total within(ms) 15
4 open index time:32
16441 total within(ms) 15
21849 total within(ms) 15
26031 total within(ms) 47
12259 total within(ms) 31
655 total within(ms) 47
5 open index time:47
19284 total within(ms) 16
25863 total within(ms) 15
30774 total within(ms) 31
14373 total within(ms) 31
730 total within(ms) 16
6 open index time:78
23073 total within(ms) 47
30892 total within(ms) 15
36732 total within(ms) 31
17233 total within(ms) 31
886 total within(ms) 31
7 open index time:31
26495 total within(ms) 16
35293 total within(ms) 15
42032 total within(ms) 47
19756 total within(ms) 47
1040 total within(ms) 62
8 open index time:63
29912 total within(ms) 31
40072 total within(ms) 31
47713 total within(ms) 47
22271 total within(ms) 46
1190 total within(ms) 31
9 open index time:47
33866 total within(ms) 31
45235 total within(ms) 31
53880 total within(ms) 47
25221 total within(ms) 47
1351 total within(ms) 31
10 open index time:47
37182 total within(ms) 31
49783 total within(ms) 31
59215 total within(ms) 47
27750 total within(ms) 63
1471 total within(ms) 31
11 open index time:47
39762 total within(ms) 46
53169 total within(ms) 31
63286 total within(ms) 63
29645 total within(ms) 94
1558 total within(ms) 47
12 open index time:109
42495 total within(ms) 63
56843 total within(ms) 31
67714 total within(ms) 63
31624 total within(ms) 94
1692 total within(ms) 47
13 open index time:31
42750 total within(ms) 32
57280 total within(ms) 31
68210 total within(ms) 63
31820 total within(ms) 62
1700 total within(ms) 31
14 open index time:32
42750 total within(ms) 31
57280 total within(ms) 31
68210 total within(ms) 47
31820 total within(ms) 63
1700 total within(ms) 31
15 open index time:47
42750 total within(ms) 32
57280 total within(ms) 31
68210 total within(ms) 47
31820 total within(ms) 94
1700 total within(ms) 31
16 open index time:62
42750 total within(ms) 32
57280 total within(ms) 31
68210 total within(ms) 47
31820 total within(ms) 46
1700 total within(ms) 31
17 open index time:31
42750 total within(ms) 94
57280 total within(ms) 32
68210 total within(ms) 47
31820 total within(ms) 47
1700 total within(ms) 31
18 open index time:31
42750 total within(ms) 31
57280 total within(ms) 31
68210 total within(ms) 47
31820 total within(ms) 46
1700 total within(ms) 32
19 open index time:31
42750 total within(ms) 32
57280 total within(ms) 16
68210 total within(ms) 63
31820 total within(ms) 62
1700 total within(ms) 32


compount index search
0 open index time:172
274 total within(ms) 156
328 total within(ms) 15
407 total within(ms) 31
195 total within(ms) 16
18 total within(ms) 16
1 open index time:31
3967 total within(ms) 16
5190 total within(ms) 0
6201 total within(ms) 15
2956 total within(ms) 32
162 total within(ms) 15
2 open index time:31
7919 total within(ms) 0
10484 total within(ms) 16
12486 total within(ms) 94
5917 total within(ms) 32
315 total within(ms) 15
3 open index time:63
12192 total within(ms) 15
16204 total within(ms) 16
19297 total within(ms) 31
9099 total within(ms) 32
485 total within(ms) 15
4 open index time:47
16048 total within(ms) 16
21349 total within(ms) 16
25419 total within(ms) 31
11978 total within(ms) 32
628 total within(ms) 47
5 open index time:31
18937 total within(ms) 16
25331 total within(ms) 0
30129 total within(ms) 31
14139 total within(ms) 32
718 total within(ms) 31
6 open index time:31
22436 total within(ms) 16
29925 total within(ms) 16
35620 total within(ms) 31
16741 total within(ms) 31
874 total within(ms) 32
7 open index time:47
25637 total within(ms) 32
34344 total within(ms) 16
40892 total within(ms) 32
19089 total within(ms) 32
1020 total within(ms) 31
8 open index time:31
29294 total within(ms) 31
39122 total within(ms) 16
46589 total within(ms) 47
21827 total within(ms) 47
1165 total within(ms) 31
9 open index time:15
31953 total within(ms) 32
42795 total within(ms) 16
50932 total within(ms) 47
23816 total within(ms) 62
1235 total within(ms) 15
10 open index time:109
34200 total within(ms) 63
45824 total within(ms) 31
54568 total within(ms) 63
25456 total within(ms) 63
1360 total within(ms) 32
11 open index time:47
37844 total within(ms) 16
50580 total within(ms) 31
60233 total within(ms) 62
28191 total within(ms) 47
1505 total within(ms) 47
12 open index time:31
40173 total within(ms) 47
53805 total within(ms) 31
64017 total within(ms) 46
29961 total within(ms) 62
1566 total within(ms) 31
13 open index time:47
42675 total within(ms) 31
57140 total within(ms) 47
68039 total within(ms) 63
31776 total within(ms) 62
1694 total within(ms) 31
14 open index time:0
42750 total within(ms) 32
57280 total within(ms) 31
68210 total within(ms) 47
31820 total within(ms) 47
1700 total within(ms) 16
15 open index time:0
42750 total within(ms) 47
57280 total within(ms) 31
68210 total within(ms) 47
31820 total within(ms) 47
1700 total within(ms) 15

trandistional index search while indexing with default minMergeDoc=100
0 open index time:141
298 total within(ms) 125
356 total within(ms) 0
442 total within(ms) 32
212 total within(ms) 46
18 total within(ms) 0
1 open index time:78
3970 total within(ms) 0
5205 total within(ms) 0
6219 total within(ms) 15
2956 total within(ms) 32
162 total within(ms) 16
2 open index time:31
8242 total within(ms) 16
10917 total within(ms) 15
13021 total within(ms) 94
6138 total within(ms) 47
332 total within(ms) 16
3 open index time:62
12533 total within(ms) 16
16686 total within(ms) 15
19889 total within(ms) 62
9330 total within(ms) 32
502 total within(ms) 31
4 open index time:93
16852 total within(ms) 16
22485 total within(ms) 0
26801 total within(ms) 32
12536 total within(ms) 31
672 total within(ms) 63
5 open index time:78
21162 total within(ms) 31
28258 total within(ms) 16
33674 total within(ms) 47
15746 total within(ms) 47
842 total within(ms) 31
6 open index time:31
25449 total within(ms) 32
34007 total within(ms) 15
40517 total within(ms) 47
18939 total within(ms) 47
1012 total within(ms) 16
7 open index time:32
29744 total within(ms) 32
39758 total within(ms) 31
47366 total within(ms) 47
22136 total within(ms) 47
1184 total within(ms) 15
8 open index time:63
33895 total within(ms) 15
45293 total within(ms) 31
53958 total within(ms) 47
25230 total within(ms) 47
1352 total within(ms) 31
9 open index time:46
37844 total within(ms) 32
50572 total within(ms) 16
60225 total within(ms) 47
28191 total within(ms) 47
1505 total within(ms) 32
10 open index time:47
40486 total within(ms) 31
54219 total within(ms) 16
64536 total within(ms) 63
30169 total within(ms) 62
1575 total within(ms) 32
11 open index time:0
42750 total within(ms) 62
57280 total within(ms) 31
68210 total within(ms) 47
31820 total within(ms) 63
1700 total within(ms) 32

-----Original Message-----
From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com] 
Sent: Thursday, June 10, 2004 4:00 AM
To: Lucene Users List
Subject: RE: Performance: compound vs. multi-file index, indexing and
searching

Hello Hui,

You are actually the person whose email I was referring to.  Have you,
by any chance, done the same test with newever versions of Lucene?  I
am using the latest RC, and see very little or no difference in
indexing speed.

Thanks,
Otis

--- hui <hui@triplehop.com> wrote:
> I did the test earlier on 1.3
>
http://issues.apache.org/eyebrowse/ReadMsg?listName=lucene-user@jakarta.apac
> he.org&msgId=1408808
> 
> 
> 
> Regards,
> Hui
> 
> -----Original Message-----
> From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com] 
> Sent: Tuesday, June 08, 2004 5:23 AM
> To: Lucene Users List
> Subject: Performance: compound vs. multi-file index, indexing and
> searching
> 
> Hello,
> 
> I was wondering if anyone can comment on the performance difference
> of
> compound versus multi-file indices.  I am interested in both indexing
> and searching performance, and have tried testing indexing
> performance
> of both formats.
> 
> My tests so far show no indexing performance differences between the
> two formats, but I do remember one person reporting compound index
> format being 7% slower, so perhaps my tests are flawed.
> 
> Can anyone comment on performance differences?
> 
> Thanks,
> Otis
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message