Return-Path: Delivered-To: apmail-lucene-java-user-archive@www.apache.org Received: (qmail 58962 invoked from network); 11 Aug 2009 16:44:37 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 11 Aug 2009 16:44:37 -0000 Received: (qmail 16779 invoked by uid 500); 11 Aug 2009 16:40:38 -0000 Delivered-To: apmail-lucene-java-user-archive@lucene.apache.org Received: (qmail 16754 invoked by uid 500); 11 Aug 2009 16:40:38 -0000 Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-user@lucene.apache.org Delivered-To: mailing list java-user@lucene.apache.org Received: (qmail 16744 invoked by uid 99); 11 Aug 2009 16:40:38 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Aug 2009 16:40:38 +0000 X-ASF-Spam-Status: No, hits=-1.0 required=10.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of jibojohn@mac.com designates 17.148.16.93 as permitted sender) Received: from [17.148.16.93] (HELO asmtpout018.mac.com) (17.148.16.93) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Aug 2009 16:40:27 +0000 MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_Ej1S5bNqpkB/SDh+PGRdgA)" Received: from la0101a-dhcp176.apple.com ([17.214.14.176]) by asmtp018.mac.com (Sun Java(tm) System Messaging Server 6.3-8.01 (built Dec 16 2008; 32bit)) with ESMTPSA id <0KO800H0V0YDOZ80@asmtp018.mac.com> for java-user@lucene.apache.org; Tue, 11 Aug 2009 09:40:07 -0700 (PDT) Cc: Michael McCandless Message-id: From: Jibo John To: java-user@lucene.apache.org In-reply-to: <9ac0c6aa0908110313p23fcb7f1uc9ee1018c7ab4256@mail.gmail.com> Subject: Re: ThreadedIndexWriter vs. IndexWriter Date: Tue, 11 Aug 2009 09:39:49 -0700 References: <9cafbc680907301111t4768704cy9e4b3092b3d74b17@mail.gmail.com> <9ac0c6aa0907310234h798d406bg8cb23d620b022ef8@mail.gmail.com> <161DCF62-1E03-4F1B-A590-76ADD91D5434@mac.com> <9ac0c6aa0907311452q4f49f1aeua7314b6059b5973@mail.gmail.com> <2BC00C69-41B3-48B3-83E7-CBC6DB452CCC@mac.com> <9cafbc680907311701g7c8ff0f1n896c4bed2adda9c9@mail.gmail.com> <090CF9D7-5D59-47A5-AD2A-86D4BEC6CD15@mac.com> <9ac0c6aa0908010208g757a9936vd1150bdec47bc9e9@mail.gmail.com> <483308B7-933B-4496-814C-0715D784836A@mac.com> <9ac0c6aa0908110313p23fcb7f1uc9ee1018c7ab4256@mail.gmail.com> X-Mailer: Apple Mail (2.935.3) X-Virus-Checked: Checked by ClamAV on apache.org --Boundary_(ID_Ej1S5bNqpkB/SDh+PGRdgA) Content-type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-transfer-encoding: 7BIT Mike, I wasn't exactly using the lucene core jar from MEAP. I have been building lucene from the source, and running the tests under lucene/java/trunk/contrib/benchmark/ (checked out 2 weeks ago, I guess) and, also under lucene/java/tags/lucene_2_4_1/contrib/ benchmark/. In both cases, copied CreateThreadedIndexTask to org.apache.lucene.benchmark.byTask.tasks and ThreadedIndexWriter to org.apache.lucene.index. I have observed the issue in both the versions of lucene. Indexes were optimized separately using Lucli. PFA the classes and the alg. --Boundary_(ID_Ej1S5bNqpkB/SDh+PGRdgA) Content-type: application/zip; x-unix-mode=0644; name=ti.zip Content-transfer-encoding: base64 Content-disposition: attachment; filename=ti.zip UEsDBAoAAAAAAJJMCzsAAAAAAAAAAAAAAAADABAAdGkvVVgMAA2egUoUnoFK9QEUAFBLAwQUAAgA CACQTAs7AAAAAAAAAAAAAAAAHwAQAHRpL0NyZWF0ZVRocmVhZGVkSW5kZXhUYXNrLmphdmFVWAwA IZ6BSg+egUr1ARQAnZJNj9owEIbv+RWjPQWETFv1hjisll0JiW2riqrnwRkSL4md2uMSttr/XofE bRBsq8UHW7Lf95kPT41yhzmBsbnAGmVBovSSNIkNaVlUaHdic1ij2wkOm5sliapqYxme8CcKZcTy 830jqWZl9Cy+ndOUzqgRy3b/bhWT/a92XVjCjLITz+ums3S/kN1+9XqBjP+IdWbzrEon7ozeqjzE m47HcAtt6cAFMhyMB4kavCPYWlMBgjSardpM/6ASGANgmZuQdFEBG5ChFKagvVCUgPE0SWq/KZUE WaJzcHeUn2jb3IAaJp3F97/3v5IEoCe84k0H7QDbnaNghLCcr8mm8XIW7l4GQKUZMrMyuZLpKDTB mr2Dwaf3kAt8mENO3F+mRzBA19m2ae0xj1oRlN1TFA5aBPvumIOm/aUepgPKQlmSbOwhHU2OoDet AehWY3l4DvBrOGw9XWHr2tJGT2+6ooX2leBjye5mAh+vSeYCtcKmp4ofnjwJp54p8D+8G02i7e2B hmP9iM2DojJbkc65EN8+rZaPy/X9Is7B6QwLRzxw96PQJTvpC+id8YtOHb02aoi91fC+G+aX5DdQ SwcItmz7EbsBAADnBAAAUEsDBAoAAAAAAJlMCzsAAAAAAAAAAAAAAAAJABAAX19NQUNPU1gvVVgM ACGegUohnoFK9QEUAFBLAwQKAAAAAACZTAs7AAAAAAAAAAAAAAAADAAQAF9fTUFDT1NYL3RpL1VY DAAhnoFKIZ6BSvUBFABQSwMEFAAIAAgAkEwLOwAAAAAAAAAAAAAAACoAEABfX01BQ09TWC90aS8u X0NyZWF0ZVRocmVhZGVkSW5kZXhUYXNrLmphdmFVWAwAIZ6BSg+egUr1ARQAjVDLSsNAFL2Jigou utGFIGSTheDUaWaaTJoipmkCLRSlFuquTOtgY/NqM7ryH/QTBH/GT/BzHGmg0pUH7pNzuJwLeyf7 oAMM+My4uTPujQq/OzhUYQFomapq1j7gX/BHo+HZ+8ttpfhS8b1F0av9G4A5y9M6L4pE1FMh+QOX vLUYdHtSpOO5WIlolaelompjlXoAxxv+8pmveCbjTMC0SOJSYvyp6TuT2mk/nuZGP59nRvtJtddr vlJeDWU8qbVTUZb8UbRMEjDXCXAYBiho0ghRnznIJayDMCWEdLFPmWtv9CYJD47OL9YuNK2ys7tl b7G8xAoe5azhigbxBjxOPOz4AcaRuhRZHURt5iMWBR3kWMRvhtiOqN14/fMNpYEfUEsHCKziyv4g AQAAoAEAAFBLAwQUAAgACACSTAs7AAAAAAAAAAAAAAAAGwAQAHRpL1RocmVhZGVkSW5kZXhXcml0 ZXIuamF2YVVYDAAhnoFKE56BSvUBFADNV9uO2zYQfddXTGGgkBeu9F43Bby2i6TY7Ca10+QtoKWx xSxNCiTlSwL/e4e6mb5tve22iB5kmRyemTmcC5mz5JEtEJReRCxnSYaRKBKUGHGZ4qYfBHyZK23h C1uxiKvozcN4k2BuuZL7ydPVxiqN0YhrTOhr278smaqkWKK00aj+eEK2tCl6494fNbeo/1Z2qLQu clsu8Qy/uIpJJraGm2jgPr5eoWGKenlEU2G5iBIlk0Jr59k008jSd0qJ8QaTggjpPynfSE1Qr3iC TwtP+RI/SG6flhpozba3QiWPXC7eF1gQahDf3MBIq/wnLkFjLliCbgNgrjR4JIPNmIXCoIFlISzP BQZwAzTs3DI9KEjWSSFkSqU9sApKboClKabQ7LCJ4CYOgryYCZ5AIpgxUFGDqa8ONxZlag5M+BYE ALnmK2YRjvipDXH89n2xSsPvagbEjChdM/BHISWbCYRvcP6J40HgfptwdOb3y5EmJIC1seGGXQBA iqIKBDdSe0iaQx+ldyDaO8XrkpuVETajEKQl8Gqvvh5thGnq0I5mVQXvVvo27XzLVoqnoAsZdi/S cI6Y20aP3raWAvA5hK1RP7wCWQjR9eYricasswJwLg4iU+T0LvKUNrPhMWzZKwltmet7aDtAYfBK eArRPfYFyArwf3HpxRzxzQ8auITZJIPQq+HAFfqGxzFMH0YPP8NWFZTtxoJQC/gR0CRMuKRyMQao tdL7AFAYUc5JO7HUTaaaykg42RqLy4gEW0N2bRzuyiytQvGMQ2HbNiDl2s+THswoyZFJSGiVxR7V GUvkL6dNKXL/l2xTVrgJ/0oSPlNv2eY3jiK9Q7mwGSznott4QTVErQ2c7RcE4jFWkVWSHpb2kVmN OQ6x8ndfkygTJa7htAmEcOGJ46G36b573rcncafkIno7+PT5z8HdhzGVmbolRPeD+4fJePhwP5r4 8s6c04bwS1Mafw0PGZwzisLuMcCpP9GQCYGaYMw7RXu7DbslGf52l5XHD1W/QF6uRnE8OmI1wlIr hs4WV2nL5HV5WL1J9VmM3SXSr3oaO/49xjV8+KF/yM3z+GCnZMD3zMdRfSyLrC0r5KVouZaPCuW7 jw/4R4Qch8vzSPlvg+TIoUQog3T6eG7VnXPJTRbWNbZqfDXW2UpTzTVNI1UfGbcvqbVGPKtbKyFm 1BBfyM093F5bfdAt1TVrrj7RxfH4ODxMVthUrWWjeZ1xgaHVhXdGOD38eQBsTXS48OSSOX/Ci62p bkvdrpcvM8J5bI8L7WlFUoA63jD1ji0Hp5aqF0X1Zaf6F3bpglavDM8eQnbuBhR0BvBaCbpuKInt VcVdYWZY3V+Czi2lGV1+CFXAWulHN0tTdCWsM7NdF3SGMCzPAjWtkBMtQWcEr9nqYAzqVIQvahZ0 xjAh6g048g9W0oXpL1BLBwg7bNhYMQQAAKMPAABQSwMEFAAIAAgAkkwLOwAAAAAAAAAAAAAAACYA EABfX01BQ09TWC90aS8uX1RocmVhZGVkSW5kZXhXcml0ZXIuamF2YVVYDAAhnoFKE56BSvUBFACN UMtKw0AUvamKFVx0owtByKYLwakTZ/KYpohJOoEWilAL1VWZ1sHG5tUmuvIf9BMEf8ZP8HMcaaDS lQfu63DO4lzYO96HGsBAzPSbW/1Or/DLwYGqSwAtVVPd2gf8C95oNDx9f7mvHF+qvrcktYp/A2jO sqQl8jyWrUSW4kGUor0YdHulTMZzuZLhKksKJdXGqvUAjjb65bNYibSMUgnTPI6KEuNPrbYzaZz0 o2mm97N5qnee1Hq91ivn1bCMJo1OIotCPMp2kwQOswPMeYACk4aIeo6NGHF8hCkhpIs96jBr428S Xj88O1+n0LQqzu5WvMXyAiu4VDgGkwZ1ByKKXc4s0zbtEHUDShHlnoF8y+aIMRZywzQJ9v3XP99Q HvgBUEsHCHI+sWghAQAAoAEAAFBLAwQUAAgACACSTAs7AAAAAAAAAAAAAAAAEgAQAHRpL3RocmVh ZGluZGV4LmFsZ1VYDAAhnoFKFJ6BSvUBFACdkUFrGzEQhe/6FcK5NIcOScEUAirYdQuFthRv7mUs zcbCK2mrkWqvg/97pPWahtJCGx3E8N684dMIPXbDkaIK8QGwR70l6LImT4DVYsvACb3BaKCZisUU EiZocLj7Y3xDXm8dxh1shnvkHbREhuGz9bQK+ktNiSsdfCKfgEOOmv57SjPGhLGRdApxUB+b1aUW csTjUpORSqaYSb7gXC3GQQyt7Ujt7c72ZCxCVxAY0iGJiA7aLvMW3EbNb4QOrg/ZG9VixyQcxQeC FiuVmo9QiaKDnyPn1PSbCn1gm2zw/Bc/tC1Turj7aIsJPjtI20hoWN3OL6rDw6TCj0yZgO2R1Nvi h7LWsjtVi+9JlI09ytm6ovPsJau6bGwphZRrKoDNwInch4hMVSvjl9l25pM3dJhVRb5+X8gS3Y+A dHZeXY9WbV8YU/654JwL+U7eyTc39UzpLjA9C53q9ZX24yukOJX22yKJNfVNdsvhW6T27P0C+feH raQQT1BLBwiCATx3eQEAADIDAABQSwMEFAAIAAgAkkwLOwAAAAAAAAAAAAAAAB0AEABfX01BQ09T WC90aS8uX3RocmVhZGluZGV4LmFsZ1VYDAAhnoFKFJ6BSvUBFABjYBVjZ2BiYPBNTFbwD1aIUIAC kBgDJxAbMTAw5gNpIJ9xIQNRwDEkJEh2QlkMVMcJIL6JpoQJKj6BgUE1OT9XL7GgICdVLze1JDEl sSTRKtvXxbMkNTc8I7Uo1a0oP7cYqJQxAkh4MjCIItQXliYWJeaVZOalMiQV5GQWlxgYLGZkYo4X kPLKTMpX8MrPyFOwyQIyHSDqgTrtghNz0uMFbHJTi4sT01OtVI2dDcwsDI0Mzd10zQ1N3HRNTF1d dC0dnV10DVzNTZyc3QxMDAxMEAaoGrty8GjqQrzByAj1Dwua/3IK9Q2AwNok0cLQMtXQ1No3MTPH 2sTV1NzZ2dxZ18LMwBRok5mLrqOJubGus4WTq5Obi5GZgaN5DVJwAPUwAABQSwcIrtk7CyMBAACh AQAAUEsBAhUDCgAAAAAAkkwLOwAAAAAAAAAAAAAAAAMADAAAAAAAAAAAQO1BAAAAAHRpL1VYCAAN noFKFJ6BSlBLAQIVAxQACAAIAJBMCzu2bPsRuwEAAOcEAAAfAAwAAAAAAAAAAECkgTEAAAB0aS9D cmVhdGVUaHJlYWRlZEluZGV4VGFzay5qYXZhVVgIACGegUoPnoFKUEsBAhUDCgAAAAAAmUwLOwAA AAAAAAAAAAAAAAkADAAAAAAAAAAAQP1BSQIAAF9fTUFDT1NYL1VYCAAhnoFKIZ6BSlBLAQIVAwoA AAAAAJlMCzsAAAAAAAAAAAAAAAAMAAwAAAAAAAAAAED9QYACAABfX01BQ09TWC90aS9VWAgAIZ6B SiGegUpQSwECFQMUAAgACACQTAs7rOLK/iABAACgAQAAKgAMAAAAAAAAAABApIG6AgAAX19NQUNP U1gvdGkvLl9DcmVhdGVUaHJlYWRlZEluZGV4VGFzay5qYXZhVVgIACGegUoPnoFKUEsBAhUDFAAI AAgAkkwLOzts2FgxBAAAow8AABsADAAAAAAAAAAAQKSBQgQAAHRpL1RocmVhZGVkSW5kZXhXcml0 ZXIuamF2YVVYCAAhnoFKE56BSlBLAQIVAxQACAAIAJJMCztyPrFoIQEAAKABAAAmAAwAAAAAAAAA AECkgcwIAABfX01BQ09TWC90aS8uX1RocmVhZGVkSW5kZXhXcml0ZXIuamF2YVVYCAAhnoFKE56B SlBLAQIVAxQACAAIAJJMCzuCATx3eQEAADIDAAASAAwAAAAAAAAAAECkgVEKAAB0aS90aHJlYWRp bmRleC5hbGdVWAgAIZ6BShSegUpQSwECFQMUAAgACACSTAs7rtk7CyMBAAChAQAAHQAMAAAAAAAA AABApIEaDAAAX19NQUNPU1gvdGkvLl90aHJlYWRpbmRleC5hbGdVWAgAIZ6BShSegUpQSwUGAAAA AAkACQDbAgAAmA0AAAAA --Boundary_(ID_Ej1S5bNqpkB/SDh+PGRdgA) Content-type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-transfer-encoding: 7BIT Thank you for your help with this one. -Jibo On Aug 11, 2009, at 3:13 AM, Michael McCandless wrote: > I'm baffled why you're losing docs w/ ThreadedIndexWriter. > > One question: your Lucene core JAR seems to be newer than the last > MEAP update. Did you update it manually? > > Also, your indexes were optimized, but your algs don't have an > optimize step -- did you separately run an optimize? > > Could you zip up the whole shebang (ThreadedIndexWriter.java, > CreateThreadedIndexTask.java, the algs) & post? Please CC me directly > so I can grab the zip file... thanks. > > Mike > > On Mon, Aug 3, 2009 at 12:37 PM, Jibo John wrote: >> Mike, >> >> Verified that I have the latest source code. >> Here are the alg files and the checkindexer output. >> >> >> ----------------------------------------- indexwriter >> alg---------------------------------------------------------------- >> >> analyzer=org.apache.lucene.analysis.standard.StandardAnalyzer >> doc.maker=org.apache.lucene.benchmark.byTask.feeds.LineDocMaker >> directory=FSDirectory >> >> doc.stored = >> true #A >> docs.file=wikipedia.lines.txt >> ram.flush.mb=50 >> compound=false >> merge.factor=5 >> doc.add.log.step=1000 >> doc.term.vector=false >> doc.term.vector.positions=false >> doc.term.vector.offsets=false >> >> { "Rounds >> " #B >> ResetSystemErase >> { "BuildIndex" >> -CreateIndex() >> [ { "AddDocs" AddDoc > : 40000 ] : >> 5 #C >> -CloseIndex() >> } >> NewRound >> } : 1 >> >> RepSumByPrefRound >> BuildIndex #D >> >> -----------------------------------------threadedindexwriter alg >> ---------------------------------------------------------------- >> >> analyzer=org.apache.lucene.analysis.standard.StandardAnalyzer >> doc.maker=org.apache.lucene.benchmark.byTask.feeds.LineDocMaker >> directory=FSDirectory >> >> doc.stored = >> true #A >> docs.file=wikipedia.lines.txt >> ram.flush.mb=50 >> compound=false >> merge.factor=5 >> doc.add.log.step=1000 >> doc.term.vector=false >> doc.term.vector.positions=false >> doc.term.vector.offsets=false >> writer.num.threads=15 >> writer.max.thread.queue.size=75 >> work.dir=work_t >> >> >> { "Rounds >> " #B >> ResetSystemErase >> { "BuildIndex" >> -CreateThreadedIndex() >> { "AddDocs" AddDoc > : 200000 >> -CloseIndex() >> } >> NewRound >> } : 1 >> >> RepSumByPrefRound >> BuildIndex #D >> >> >> -----------------------------------------------threadedindexwriter >> checkindex ---------------------------------------------------------- >> >> >> $ java -classpath >> /Users/jibo/Desktop/iwork/lucene/java/trunk/build/lucene-core-2.9- >> dev.jar >> org.apache.lucene.index.CheckIndex >> /Users/jibo/Desktop/iwork/lucene/java/trunk/contrib/benchmark/ >> work_t/index >> >> NOTE: testing will be more thorough if you run java with >> '-ea:org.apache.lucene...', so assertions are enabled >> >> Opening index @ >> /Users/jibo/Desktop/iwork/lucene/java/trunk/contrib/benchmark/ >> work_t/index >> >> Segments file=segments_3 numSegments=1 version=FORMAT_DIAGNOSTICS >> [Lucene >> 2.9] >> 1 of 1: name=_p docCount=199941 >> compound=true >> hasProx=true >> numFiles=3 >> size (MB)=317.1 >> diagnostics = {java.version=1.5.0_19, lucene.version=2.9-dev >> 779767M - >> 2009-05-28 17:02:17, os=Mac OS X, os.arch=i386, optimize=true, >> mergeDocStores=false, java.vendor=Apple Inc., os.version=10.5.7, >> source=merge, mergeFactor=5} >> docStoreOffset=0 >> docStoreSegment=_0 >> docStoreIsCompoundFile=false >> no deletions >> test: open reader.........OK >> test: fields, norms.......OK [4 fields] >> test: terms, freq, prox...OK [1269552 terms; 67887116 terms/docs >> pairs; >> 133241176 tokens] >> test: stored fields.......OK [199941 total field count; avg 1 >> fields per >> doc] >> test: term vectors........OK [0 total vector count; avg 0 term/ >> freq vector >> fields per doc] >> >> No problems were detected with this index. >> >> ------------------------------------------indexwriter checkindex >> --------------------------------------------------------------- >> >> $ java -classpath >> /Users/jibo/Desktop/iwork/lucene/java/trunk/build/lucene-core-2.9- >> dev.jar >> org.apache.lucene.index.CheckIndex >> /Users/jibo/Desktop/iwork/lucene/java/trunk/contrib/benchmark/work/ >> index >> >> NOTE: testing will be more thorough if you run java with >> '-ea:org.apache.lucene...', so assertions are enabled >> >> Opening index @ >> /Users/jibo/Desktop/iwork/lucene/java/trunk/contrib/benchmark/work/ >> index >> >> Segments file=segments_a numSegments=1 version=FORMAT_DIAGNOSTICS >> [Lucene >> 2.9] >> 1 of 1: name=_18 docCount=200000 >> compound=true >> hasProx=true >> numFiles=1 >> size (MB)=427.445 >> diagnostics = {java.version=1.5.0_19, lucene.version=2.9-dev >> 779767M - >> 2009-05-28 17:02:17, os=Mac OS X, os.arch=i386, optimize=true, >> mergeDocStores=true, java.vendor=Apple Inc., os.version=10.5.7, >> source=merge, mergeFactor=4} >> no deletions >> test: open reader.........OK >> test: fields, norms.......OK [4 fields] >> test: terms, freq, prox...OK [3512343 terms; 80020204 terms/docs >> pairs; >> 163219760 tokens] >> test: stored fields.......OK [200000 total field count; avg 1 >> fields per >> doc] >> test: term vectors........OK [0 total vector count; avg 0 term/ >> freq vector >> fields per doc] >> >> No problems were detected with this index. >> >> --------------------------------------------------------------------------------------------------------- >> >> >> >> >> Thanks, >> -Jibo >> >> >> >> >> >> >> >> On Aug 1, 2009, at 2:08 AM, Michael McCandless wrote: >> >>> (Please note that ThreadedIndexWriter is source code available with >>> the upcoming revision to Lucene in Action.) >>> >>> Phil, is it possible you are using an older version of the book's >>> source code? In particular, can you check whether your version of >>> ThreadedIndexWriter.java has this: >>> >>> public void close(boolean doWait) throws CorruptIndexException, >>> IOException { >>> finish(); >>> super.close(doWait); >>> } >>> >>> (I vaguely remember that being missing from earlier releases, which >>> could explain what you're seeing). If you are missing that, can you >>> download the current code from http://www.manning.com/hatcher3 and >>> try >>> again? >>> >>> If that's not the problem... can you post the benchmark alg you are >>> using in each case? >>> >>> Mike >>> >>> On Fri, Jul 31, 2009 at 8:26 PM, Jibo John wrote: >>>> >>>> Hi Phil, >>>> >>>> It's 5 threads for IndexWriter. >>>> >>>> For ThreadedIndexWriter, I used: >>>> >>>> writer.num.threads=16 >>>> writer.max.thread.queue.size=80 >>>> >>>> Thanks, >>>> -Jibo >>>> >>>> On Jul 31, 2009, at 5:01 PM, Phil Whelan wrote: >>>> >>>>> Hi Jibo, >>>>> >>>>> Your mergeFactor is different, and the resulting numFiles (segment >>>>> files) is different. Maybe each thread is responsible for a >>>>> segment >>>>> file. Just curious - do you have 3 threads? >>>>> >>>>> Phil >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org >>>>> For additional commands, e-mail: java-user-help@lucene.apache.org >>>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org >>>> For additional commands, e-mail: java-user-help@lucene.apache.org >>>> >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org >>> For additional commands, e-mail: java-user-help@lucene.apache.org >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org >> For additional commands, e-mail: java-user-help@lucene.apache.org >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org > For additional commands, e-mail: java-user-help@lucene.apache.org > --Boundary_(ID_Ej1S5bNqpkB/SDh+PGRdgA) Content-Type: text/plain; charset=us-ascii --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org For additional commands, e-mail: java-user-help@lucene.apache.org --Boundary_(ID_Ej1S5bNqpkB/SDh+PGRdgA)--