lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless" <>
Subject Re: &#22238;&#22797;&#65306; DocumentsWriter questions
Date Sun, 04 Nov 2007 14:04:56 GMT

"Leon" <> wrote:

> Hmm. yes, i am proposing a new class to manage these ThreadState. I already extracted

> these code into one class called ThreadStatePool in my workplace. which somthing like:
>  private class ThreadStatePool{
>    //Max # ThreadState instances; if there are more threads
>    // than this they share ThreadStates
>    private final static int MAX_THREAD_STATE = 5;
>    private ThreadState[] threadStates = new ThreadState[0];
>    private final HashMap threadBindings = new HashMap();
>    private int numWaiting;
>    private ThreadState[] waitingThreadStates = new ThreadState[1];
>    public void restPostings() throws IOException {}
>    public synchronized boolean allThreadsIdle() {}
>    public void forceRemoveWaiting() {}// Forcefully remove waiting ThreadStates from
>    public ArrayList gatherHasPostings(){}
>    public void clearVectorsAndFields()throws IOException{}// Clear vectors & fields
from ThreadStates
>    public void sweep() throws IOException{}//If any states were waiting on me, sweep
through and flush those that are enabled by my write.
>    synchronized ThreadState getThreadState(Document doc, Term delTerm) throws IOException
> }

This looks good!  A few questions:

  * Is this a strict refactoring, or, are there other changes here?
    Eg, I'm not sure why there is a forceRemoveWaiting()?

  * Why is "resetPostings()", and "clearVectorsAndFields()" part of
    this class?  It seems like that should stay in ThreadState?
    Whereas this class is concerned with managing the set of

  * Shouldn't there also be a method to return a now-idle ThreadState
    back into the pool?

Also, you have to be very careful with locking; currently we
synchronize on the DocumentsWriter instance for many of the above
methods, eg in order to ensure that eg we can correctly idle all
threads while a flush is happening.  It's probably simplest to leave
synchronization on the DocumentsWriter instance.

> Another question, there are lots of similar operations such as reSize() the allFieldDataArray/docFieldDataArray
and so on. 
> why not make a method like:
>  void reSize(FieldData[] orign, int newSize){
>      FieldData newArray[] = new FieldData[newSize];
>      System.arraycopy(orign, 0, newArray, 0, orign.length);
>      orign = newArray;
> }


Can you work these out into a patch and open a Jira issue?  We can
iterate from there.  Thanks!


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message