Return-Path: Mailing-List: contact lucene-dev-help@jakarta.apache.org; run by ezmlm Delivered-To: mailing list lucene-dev@jakarta.apache.org Received: (qmail 44532 invoked from network); 15 Sep 2003 21:18:14 -0000 Received: from unknown (HELO jive.servlets.net) (209.162.192.250) by daedalus.apache.org with SMTP; 15 Sep 2003 21:18:14 -0000 Received: from jivesoftware.com (CPE00045ade8fc4-CM008037bbf5fd.cpe.net.cable.rogers.com [24.153.2.131]) (authenticated bits=0) by jive.servlets.net (8.12.8/8.12.8) with ESMTP id h8FLIHek023544 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Mon, 15 Sep 2003 14:18:20 -0700 Message-ID: <3F662C94.4090000@jivesoftware.com> Date: Mon, 15 Sep 2003 17:18:12 -0400 From: Bruce Ritchie Organization: Jive Software User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4.1) Gecko/20030906 X-Accept-Language: en-ca, en MIME-Version: 1.0 To: Lucene Developers List Subject: Re: Caching filter wrapper (was Re: RE : DateFilter.Before/After) References: <2AB73DBC-E796-11D7-9CE9-000393A564E6@ehatchersolutions.com> <3F65ECD3.7050302@jivesoftware.com> <3F65F349.7050100@lucene.com> <3F65F888.8010504@jivesoftware.com> <3F65FE67.7000008@lucene.com> In-Reply-To: <3F65FE67.7000008@lucene.com> Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms050105000306010406050408" X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N --------------ms050105000306010406050408 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Doug Cutting wrote: > If you do get a chance to look into this, I'd love to hear more. > > FieldsReader.doc() could easily be re-written to be re-entrant. For a > start, it could synchronize separately on fieldStream and indexStream, > which would let two threads use it at once. (If an index is not > optimized, the situtation would be even better, since there would be a > fieldStream and indexStream per index segment.) > > If that's not enough, then it could be re-written to use either a pool > of cloned input streams, or just to clone a new stream for each call. > (The primary expense of cloning a stream is allocating a 1k buffer.) I think that using a pool of cloned inputStreams would be the best solution. I've implemented such a solution locally using two pools of 3 readers each (configurable via system properties) and will post the diff after I do some testing to confirm accuracy and speed improvements. I'm a little unsure of how exactly to best write JUnit test(s) for these changes as any tests should take into account multiple threads and simultaneous reads (to test that the object pool I wrote actually works as advertised). Any ideas would be appreciated. Regards, Bruce Ritchie --------------ms050105000306010406050408 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIJWDCC AwowggJzoAMCAQICAwht7zANBgkqhkiG9w0BAQQFADCBkjELMAkGA1UEBhMCWkExFTATBgNV BAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3duMQ8wDQYDVQQKEwZUaGF3dGUx HTAbBgNVBAsTFENlcnRpZmljYXRlIFNlcnZpY2VzMSgwJgYDVQQDEx9QZXJzb25hbCBGcmVl bWFpbCBSU0EgMjAwMC44LjMwMB4XDTAyMTAwNzIwNTMyOFoXDTAzMTAwNzIwNTMyOFowSDEf MB0GA1UEAxMWVGhhd3RlIEZyZWVtYWlsIE1lbWJlcjElMCMGCSqGSIb3DQEJARYWYnJ1Y2VA aml2ZXNvZnR3YXJlLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAN1qo82c agU3foe/NdBpl0qE2gtfjKV21zQM6NjXgfi4saTNwMvPJCpoU4KXCQZC+sPOll23eo0dtugN q/70apl32G7TwckV24aL0MYbWZhr8s75k6OTfGS57w8J2T60Zi7BDEEUv8vjUG/eRYV8pSK1 3Acix+FPhN8sW+NFF0Hfe9z7brpJ/7BTsiHcEXwdqlR/XjPgcNpCbQ/QQJkf3Zmx3XYBfn7T 3fyhSm4uhPNkz941HTblz6+VBB/ICubucm+Pz5gSpHYGBm+J8jnimVMQEX1LL/wInN7vwutu sIMLoQ8C1YAQMUCnITEA38kg5ZHBwD0I/odURGXdKCjAdlECAwEAAaMzMDEwIQYDVR0RBBow GIEWYnJ1Y2VAaml2ZXNvZnR3YXJlLmNvbTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBBAUA A4GBAA+U/df/1mU5jG7Ksuc2wX+V/oUt6telqLGQNn/BdKJlsUTDFHvs2QUeaeWwmbNsbXiC JSowxA4lrUZaLdj8NZcjhdZhFqJpU/ksyem45qRYYHVMnWpoII7dSUmLQWntM70497rSUUD2 CSwJ4JfceLLDO0PKOzlCR+gmbYFTJzrdMIIDCjCCAnOgAwIBAgIDCG3vMA0GCSqGSIb3DQEB BAUAMIGSMQswCQYDVQQGEwJaQTEVMBMGA1UECBMMV2VzdGVybiBDYXBlMRIwEAYDVQQHEwlD YXBlIFRvd24xDzANBgNVBAoTBlRoYXd0ZTEdMBsGA1UECxMUQ2VydGlmaWNhdGUgU2Vydmlj ZXMxKDAmBgNVBAMTH1BlcnNvbmFsIEZyZWVtYWlsIFJTQSAyMDAwLjguMzAwHhcNMDIxMDA3 MjA1MzI4WhcNMDMxMDA3MjA1MzI4WjBIMR8wHQYDVQQDExZUaGF3dGUgRnJlZW1haWwgTWVt YmVyMSUwIwYJKoZIhvcNAQkBFhZicnVjZUBqaXZlc29mdHdhcmUuY29tMIIBIjANBgkqhkiG 9w0BAQEFAAOCAQ8AMIIBCgKCAQEA3WqjzZxqBTd+h7810GmXSoTaC1+MpXbXNAzo2NeB+Lix pM3Ay88kKmhTgpcJBkL6w86WXbd6jR226A2r/vRqmXfYbtPByRXbhovQxhtZmGvyzvmTo5N8 ZLnvDwnZPrRmLsEMQRS/y+NQb95FhXylIrXcByLH4U+E3yxb40UXQd973Ptuukn/sFOyIdwR fB2qVH9eM+Bw2kJtD9BAmR/dmbHddgF+ftPd/KFKbi6E82TP3jUdNuXPr5UEH8gK5u5yb4/P mBKkdgYGb4nyOeKZUxARfUsv/Aic3u/C626wgwuhDwLVgBAxQKchMQDfySDlkcHAPQj+h1RE Zd0oKMB2UQIDAQABozMwMTAhBgNVHREEGjAYgRZicnVjZUBqaXZlc29mdHdhcmUuY29tMAwG A1UdEwEB/wQCMAAwDQYJKoZIhvcNAQEEBQADgYEAD5T91//WZTmMbsqy5zbBf5X+hS3q16Wo sZA2f8F0omWxRMMUe+zZBR5p5bCZs2xteIIlKjDEDiWtRlot2Pw1lyOF1mEWomlT+SzJ6bjm pFhgdUydamggjt1JSYtBae0zvTj3utJRQPYJLAngl9x4ssM7Q8o7OUJH6CZtgVMnOt0wggM4 MIICoaADAgECAhBmRXK3zHT1z2N2RYTQLpEBMA0GCSqGSIb3DQEBBAUAMIHRMQswCQYDVQQG EwJaQTEVMBMGA1UECBMMV2VzdGVybiBDYXBlMRIwEAYDVQQHEwlDYXBlIFRvd24xGjAYBgNV BAoTEVRoYXd0ZSBDb25zdWx0aW5nMSgwJgYDVQQLEx9DZXJ0aWZpY2F0aW9uIFNlcnZpY2Vz IERpdmlzaW9uMSQwIgYDVQQDExtUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgQ0ExKzApBgkq hkiG9w0BCQEWHHBlcnNvbmFsLWZyZWVtYWlsQHRoYXd0ZS5jb20wHhcNMDAwODMwMDAwMDAw WhcNMDQwODI3MjM1OTU5WjCBkjELMAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2Fw ZTESMBAGA1UEBxMJQ2FwZSBUb3duMQ8wDQYDVQQKEwZUaGF3dGUxHTAbBgNVBAsTFENlcnRp ZmljYXRlIFNlcnZpY2VzMSgwJgYDVQQDEx9QZXJzb25hbCBGcmVlbWFpbCBSU0EgMjAwMC44 LjMwMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDeMzKmY8cJJUU+0m54J2eBxdqIGYKX DuNEKYpjNSptcDz63K737nRvMLwzkH/5NHGgo22Y8cNPomXbDfpL8dbdYaX5hc1VmjUanZJ1 qCeu2HL5ugL217CR3hzpq+AYA6h8Q0JQUYeDPPA5tJtUihOH/7ObnUlmAC0JieyUa+mhaQID AQABo04wTDApBgNVHREEIjAgpB4wHDEaMBgGA1UEAxMRUHJpdmF0ZUxhYmVsMS0yOTcwEgYD VR0TAQH/BAgwBgEB/wIBADALBgNVHQ8EBAMCAQYwDQYJKoZIhvcNAQEEBQADgYEAMbFLR135 AXHl9VNsXXnWPZjAJhNigSKnEvgilegbSbcnewQ5uvzm8iTrkfq97A0qOPdQVahs9w2tTBu8 A/S166JHn2yiDFiNMUIJEWywGmnRKxKyQF1q+XnQ6i4l3Yrk/NsNH50C81rbyjz2ROomaYd/ SJ7OpZ/nhNjJYmKtBcYxggPVMIID0QIBATCBmjCBkjELMAkGA1UEBhMCWkExFTATBgNVBAgT DFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3duMQ8wDQYDVQQKEwZUaGF3dGUxHTAb BgNVBAsTFENlcnRpZmljYXRlIFNlcnZpY2VzMSgwJgYDVQQDEx9QZXJzb25hbCBGcmVlbWFp bCBSU0EgMjAwMC44LjMwAgMIbe8wCQYFKw4DAhoFAKCCAg8wGAYJKoZIhvcNAQkDMQsGCSqG SIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMDMwOTE1MjExODEyWjAjBgkqhkiG9w0BCQQxFgQU LYhIz6fA18G2T9zIDXY4wr6nuUAwUgYJKoZIhvcNAQkPMUUwQzAKBggqhkiG9w0DBzAOBggq hkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgwgasG CSsGAQQBgjcQBDGBnTCBmjCBkjELMAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2Fw ZTESMBAGA1UEBxMJQ2FwZSBUb3duMQ8wDQYDVQQKEwZUaGF3dGUxHTAbBgNVBAsTFENlcnRp ZmljYXRlIFNlcnZpY2VzMSgwJgYDVQQDEx9QZXJzb25hbCBGcmVlbWFpbCBSU0EgMjAwMC44 LjMwAgMIbe8wga0GCyqGSIb3DQEJEAILMYGdoIGaMIGSMQswCQYDVQQGEwJaQTEVMBMGA1UE CBMMV2VzdGVybiBDYXBlMRIwEAYDVQQHEwlDYXBlIFRvd24xDzANBgNVBAoTBlRoYXd0ZTEd MBsGA1UECxMUQ2VydGlmaWNhdGUgU2VydmljZXMxKDAmBgNVBAMTH1BlcnNvbmFsIEZyZWVt YWlsIFJTQSAyMDAwLjguMzACAwht7zANBgkqhkiG9w0BAQEFAASCAQCBr+/wn6F4pw8kiFOL H6m94nZXHa9M4vVvfX7Q9rnUX2zV1DrEbhs0EIVeq1Q+5nvSjRiMH2E7Xn/G7bRz9iKsOd80 uisX4rMUNBFBRR8agKnpgUno+NisOaDQXHyjC7aWIkFsviG/jXL17+h/vjMyYTV3qJcnpMT9 eoCZ4TGaINZ9Tkbdfz1Pt5EehfeDYzOuN0aPMNeUyw/s0yF3d4nMWIfSlH2y1fZEHgpUbaM4 T0aKNyNTxV8Lf3WOCC5wJCKotkbJXrV1CM9V61Rv/jRz+5giNS0uCBC+5mI4RVro57ohptNV 3XCiYIdXSCX8MwHNjkMymAsoFcpNvCSZfj/kAAAAAAAA --------------ms050105000306010406050408--