lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan McKinley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-2649) FieldCache should include a BitSet for matching docs
Date Sun, 26 Sep 2010 21:17:33 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12915039#action_12915039
] 

Ryan McKinley commented on LUCENE-2649:
---------------------------------------

Thanks Mike, I'll take a look and fix the small issues later tonight.

bq. I think we should hold off on backport to 3.x until we stabilize LUCENE-2665?

+1, I think it makes sense to backport to 3.x when there is a clear upgrade path.  



bq. I think we have a synchronization issue on the call to validate?

Looks like it.  There are two approaches we could take.  

1. synchronize from the cache:
{code:java}
      if( key.creator.shouldValidate() ) {
        synchronized( key ) {
          key.creator.validate( (T)value, reader);
        }
      }
{code}

2. make each creator responcible for validation.  For example, the DoubleValuesEntryCreator
would look like:
{code:java}

  @Override
  public DoubleValues validate(DoubleValues entry, IndexReader reader) throws IOException
{
    boolean ok = false;
    if( hasOption(OPTION_CACHE_VALUES) ) {
      ok = true;
      if( entry.values == null ) {
        synchronized( this ) {
          if( entry.values == null ) {
            fillDoubleValues(entry, reader, field);
          }
        }
      }
    }
    if( hasOption(OPTION_CACHE_BITS) ) {
      ok = true;
      if( entry.valid == null ) {
        synchronized( this ) {
          if( entry.valid == null ) {
            fillValidBits(entry, reader, field);
          }
        }
      }
    }
    if( !ok ) {
      throw new RuntimeException( "the config must cache values and/or bits" );
    }
    return entry;
  }
{code}
That is a bit more complicated, but avoids synchonization when things are valid.  Thoughts?


bq. I think it's best if whoever gets there first, wins, for this case

Yes, this is the current behavior -- the validate method does not do anything:
{code:java}
  public T validate(T entry, IndexReader reader) throws IOException {
    // TODO? nothing? perhaps subsequent call with FASTER_BUT_MORE_RAM?
    return entry;
  }
{code}


- - - - - - - -

The other key behavior that we should note is that with CachedArrayEntryCreators, if you pass
a different parser for the same field, it will give you an error.  Previously this created
two cache entries (and then added something to the insanity log).  If someone wants to do
that, they could override the XxxEntryCreator.getCacheKey() to return key that includes the
Parser.  By default this seems like an error to me.





> FieldCache should include a BitSet for matching docs
> ----------------------------------------------------
>
>                 Key: LUCENE-2649
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2649
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>            Reporter: Ryan McKinley
>            Assignee: Ryan McKinley
>             Fix For: 4.0
>
>         Attachments: LUCENE-2649-FieldCacheWithBitSet.patch, LUCENE-2649-FieldCacheWithBitSet.patch,
LUCENE-2649-FieldCacheWithBitSet.patch, LUCENE-2649-FieldCacheWithBitSet.patch, LUCENE-2649-FieldCacheWithBitSet.patch,
LUCENE-2649-FieldCacheWithBitSet.patch
>
>
> The FieldCache returns an array representing the values for each doc.  However there
is no way to know if the doc actually has a value.
> This should be changed to return an object representing the values *and* a BitSet for
all valid docs.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message