I'm looking at StopFilter.java right now...

I did a kill -3 java and a number of my threads were blocked here:

"ksa-task-thread-34" prio=1 tid=0xad89fbe8 nid=0x1c6e waiting for 
monitor entry [b9bff000..b9bff8d0]
        at java.util.Hashtable.get(Hashtable.java:332)
        - waiting to lock <0x61569720> (a java.util.Hashtable)
        at org.apache.lucene.analysis.StopFilter.next(StopFilter.java:94)
        at 
org.apache.lucene.index.DocumentWriter.invertDocument(DocumentWriter.java:170)
        at 
org.apache.lucene.index.DocumentWriter.addDocument(DocumentWriter.java:111)
        at 
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:257)
        at 
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:244)
        at 
ksa.index.AdvancedIndexWriter.addDocument(AdvancedIndexWriter.java:136)
        at 
ksa.robot.FeedTaskParserListener.onItemEnd(FeedTaskParserListener.java:331)

Is there ANY reason to keep this as a Hashtable?  It's just preventing 
inversion across multiple threads.  They all have to lock on this hashtable.

Note that this guy is initialized ONCE and no more puts take place so I 
don't see why not.  It's readonly after the StopFilter is created.

I think this might really end up speeding up indexing a bit.  No hard 
benchmarks yet though.  Right now though it's just an inefficiency that 
should be removed.

I've attached a quick implementation. 

Kevin

-- 

Please reply using PGP:

    http://peerfear.org/pubkey.asc    

    NewsMonster - http://www.newsmonster.org/
    
Kevin A. Burton, Location - San Francisco, CA, Cell - 415.595.9965
       AIM/YIM - sfburtonator,  Web - http://peerfear.org/
GPG fingerprint: 5FB2 F3E2 760E 70A8 6174 D393 E84D 8D04 99F1 4412
  IRC - freenode.net #infoanarchy | #p2p-hackers | #newsmonster