Mailing-List: contact lucene-dev-help@jakarta.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Lucene Developers List" <lucene-dev@jakarta.apache.org>
Mime-Version: 1.0 (Apple Message framework v612)
In-Reply-To: <404CD6AB.6070707@apache.org>
References: <20040308193436.77055.qmail@web12706.mail.yahoo.com>
 <237201c40545$c7959d90$6501a8c0@POWERPACK> <404CD6AB.6070707@apache.org>
Content-Type: multipart/signed; micalg=sha1;
 boundary=Apple-Mail-57--453129786; protocol="application/pkcs7-signature"
Message-Id: <0072E3C5-7152-11D8-B209-000A95D01A94@ganyo.com>
From: Scott ganyo <scott@ganyo.com>
Subject: Re: compound format as default in 1.4?
Date: Mon, 8 Mar 2004 17:43:21 -0500
To: "Lucene Developers List" <lucene-dev@jakarta.apache.org>

--Apple-Mail-57--453129786
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
	charset=US-ASCII;
	delsp=yes;
	format=flowed

+1.  I agree with this.  Give the safest option to the general masses,  
let the "expert" users choose other options based on their level of  
experience.

(BTW:  It seems that accessing this compound file format as a memory  
mapped file using the NIO library would be a natural fit for improving  
Lucene's memory footprint as well...)

Scott

On Mar 8, 2004, at 3:25 PM, Doug Cutting wrote:

> [ I moved this discussion to the developer list.]
>
> My metric here is the rate of complaint.
>
> I'm tired of hearing about "too many file handles" problems.  Ususally  
> it is caused by folks opening a new searcher for each query, and the  
> garbage collector not collecting and closing the old ones fast enough,  
> so it signals other problems with the application, but it is still  
> annoying, and could be largely quashed.
>
> By some definition, anything which causes so many repeated complaints  
> is a bug, and should be fixed.  Even if it's really not a bug.  It  
> pains users of Lucene.  It annoys developers of Lucene.
>
> Think of it like mergeFactor, etc.: the default setting may not be the  
> absolute fastest, but it is one that is likely to run well in most  
> configurations and cause the least confusion.
>
> Doug
>
> Terry Steichen wrote:
>> I tend to agree (but with the same uncertainty as to why I feel that  
>> way).
>> Regards,
>> Terry
>> ----- Original Message ----- From: "Otis Gospodnetic"  
>> <otis_gospodnetic@yahoo.com>
>> To: "Lucene Users List" <lucene-user@jakarta.apache.org>
>> Sent: Monday, March 08, 2004 2:34 PM
>> Subject: Re: Sys properties Was: java.io.tmpdir as lock dir .... once  
>> again
>>> I can't explain why, but I feel like the old index format should stay
>>> by default.  I feel like I'd rather a (slightly) faster index, and
>>> switch to the compound one when/IF I encounter problems, than have a
>>> safer, but slower index, and never realize that there is a faster
>>> option available.
>>>
>>> Weak argument, I know, but some instinct in me thinks that the  
>>> current
>>> mode should remain.
>>>
>>> Otis
>>>
>>>
>>> --- Doug Cutting <cutting@apache.org> wrote:
>>>
>>>> hui wrote:
>>>>
>>>>> Index time: compound format is 89 seconds slower.
>>>>>
>>>>> compound format:
>>>>> 1389507 total milliseconds
>>>>> non-compound format:
>>>>> 1300534 total milliseconds
>>>>>
>>>>> The index size is 85m with 4 fields only. The files are stored in
>>>>
>>>> the index.
>>>>
>>>>> The compound format has only 3 files and the other has 13 files.
>>>>
>>>> Thanks for performing this benchmark!
>>>>
>>>> It looks like the compound format is around 7% slower when  
>>>> indexing. To my thinking that's acceptable, given the dramatic  
>>>> reduction in file handles.  If folks really need maximal indexing  
>>>> performance, then
>>>> they can explicitly disable the compound format.
>>>>
>>>> Would anyone object to making compound format the default for  
>>>> Lucene 1.4?  This is an incompatible change, but I don't think it  
>>>> should
>>>> break applications.
>>>>
>>>> Doug
>>>>
>>>> -------------------------------------------------------------------- 
>>>> -
>>>> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
>>>> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
>

--Apple-Mail-57--453129786
Content-Transfer-Encoding: base64
Content-Type: application/pkcs7-signature;
	name=smime.p7s
Content-Disposition: attachment;
	filename=smime.p7s

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIGEjCCAssw
ggI0oAMCAQICAwtKHTANBgkqhkiG9w0BAQQFADBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhh
d3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVt
YWlsIElzc3VpbmcgQ0EwHhcNMDMxMjA4MTk0NTI5WhcNMDQxMjA3MTk0NTI5WjBBMR8wHQYDVQQD
ExZUaGF3dGUgRnJlZW1haWwgTWVtYmVyMR4wHAYJKoZIhvcNAQkBFg9zY290dEBnYW55by5jb20w
ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC2ptQeyZPoLu2i4pPIHtrUiJ05HhAcXGyf
qMsrhx45ToVW4qrKmBNBfARFOe+PWiYUun8U7wKi3GmVWfxvQ7yqeiUu6DnLWMVGmFgx8xZQiIG2
6aa9to5xGL/SOTo7ET2+cSx7zX7CKiFSMALKrVzCXiAunGFp6ONkpqOCMj3Mk0U8QxFIkN3yuTO+
f7a9VH0tM6SPjyOWnR6SMbtU6WBlfO4JWCKnE2u1zzh6vOiYfjiKRpWYkaZu9BCYHAYHNwAHosT5
JaaY/+hV7QSlxHERz08kQv9V3K6PRIr6Ey68BeZnQbZp6E3ua9DWmlZL9gnbRPDob0kMbNiBJTkW
Zo9fAgMBAAGjLDAqMBoGA1UdEQQTMBGBD3Njb3R0QGdhbnlvLmNvbTAMBgNVHRMBAf8EAjAAMA0G
CSqGSIb3DQEBBAUAA4GBAFEM8YLGdg6q71DscLXqUrklI8I3OQ3KpkWMrPHiAnMH3ZG82xyGMe/v
/pRCT60gaC9xLoRi8AmMeUB8ykcVsgGi5YfrO2k4LbFR52tYk9fg0OU7VtpGZW0ehJuiuPrIupX8
R58ex+L0qZcGQ7c/f1ZyGZiaDNV/iQneGskM8p8wMIIDPzCCAqigAwIBAgIBDTANBgkqhkiG9w0B
AQUFADCB0TELMAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2Fw
ZSBUb3duMRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEoMCYGA1UECxMfQ2VydGlmaWNhdGlv
biBTZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIENB
MSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVlbWFpbEB0aGF3dGUuY29tMB4XDTAzMDcxNzAw
MDAwMFoXDTEzMDcxNjIzNTk1OVowYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25z
dWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1
aW5nIENBMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDEpjxVc1X7TrnKmVoeaMB1BHCd3+n/
ox7svc31W/Iadr1/DDph8r9RzgHU5VAKMNcCY1osiRVwjt3J8CuFWqo/cVbLrzwLB+fxH5E2JCoT
zyvV84J3PQO+K/67GD4Hv0CAAmTXp6a7n2XRxSpUhQ9IBH+nttE8YQRAHmQZcmC3+wIDAQABo4GU
MIGRMBIGA1UdEwEB/wQIMAYBAf8CAQAwQwYDVR0fBDwwOjA4oDagNIYyaHR0cDovL2NybC50aGF3
dGUuY29tL1RoYXd0ZVBlcnNvbmFsRnJlZW1haWxDQS5jcmwwCwYDVR0PBAQDAgEGMCkGA1UdEQQi
MCCkHjAcMRowGAYDVQQDExFQcml2YXRlTGFiZWwyLTEzODANBgkqhkiG9w0BAQUFAAOBgQBIjNFQ
g+oLLswNo2asZw9/r6y+whehQ5aUnX9MIbj4Nh+qLZ82L8D0HFAgk3A8/a3hYWLD2ToZfoSxmRsA
xRoLgnSeJVCUYsfbJ3FXJY3dqZw5jowgT2Vfldr394fWxghOrvbqNOUQGls1TXfjViF4gtwhGTXe
JLHTHUb/XV9lTzGCAucwggLjAgEBMGkwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBD
b25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJ
c3N1aW5nIENBAgMLSh0wCQYFKw4DAhoFAKCCAVMwGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAc
BgkqhkiG9w0BCQUxDxcNMDQwMzA4MjI0MzIxWjAjBgkqhkiG9w0BCQQxFgQUuoz1ibI9LfMnd784
358vziG1TUYweAYJKwYBBAGCNxAEMWswaTBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3Rl
IENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWls
IElzc3VpbmcgQ0ECAwtKHTB6BgsqhkiG9w0BCRACCzFroGkwYjELMAkGA1UEBhMCWkExJTAjBgNV
BAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25h
bCBGcmVlbWFpbCBJc3N1aW5nIENBAgMLSh0wDQYJKoZIhvcNAQEBBQAEggEAfC22MOGlS4rcN+bM
29dAGngWj0Y8+GwhzjcwFgkY0c/rjM1knAlBrivey3dR9rF3ggN4ORBqQyEdutzYlhGfSdBFfs6E
iJeWxsS8nIMig+By9Yvcbj5M4QgAKHIpS03J2ZBFecKWhjpCQiVsvD2xL4IEBOY84NtXKOKtthgF
QYO7bTEbuL7OElslF4u6BVhkJa12uVYZMAxPIp7j1faiHv2flxzC3+thMs/SvDgciC8llQMI5qsk
zoMISLzzndPkmYokWjkNxzZTE/sAKeYFcyBFXaKq461PMzpQzHSfEPGAoDqnA8tJd+BvuRigAqOe
KxUHuAhzHla1Kn7Nzq254QAAAAAAAA==

--Apple-Mail-57--453129786--