lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Henson (JIRA)" <>
Subject [jira] Created: (SOLR-1787) CachedSqlEntityProcessor pre-warmed cache use case
Date Mon, 22 Feb 2010 23:08:27 GMT
CachedSqlEntityProcessor pre-warmed cache use case

                 Key: SOLR-1787
             Project: Solr
          Issue Type: Improvement
          Components: contrib - DataImportHandler
    Affects Versions: 1.5
         Environment: jdk 1.6.x, windows xp, tomcat 6.x
            Reporter: Michael Henson
            Priority: Minor
             Fix For: 1.5

The CachedSqlEntityProcessor currently builds a cache of rows it sees as it goes, so later
requests for that same key can be served from data that has already been fetched. The primary
query could be written to fetch all possible rows, which would then be set into the cache
on the first request for a row. In that case the database would only receive another query
when there is a cache miss. However, the query it would execute is the one that pulls all
rows, negating any performance gain.

This patch adds the ability to configure behavior on cache miss with the "onCacheMiss" attribute
on an "entity" tag in the data-config.xml file. The current behavior is the default, corresponding
to the setting onCacheMiss="fill". Any other value explicitly given for onCacheMiss will cause
cache misses to be ignored - no query will be made to the db to fulfill them.

I've encountered two cases where this capability is useful:

1. Relatively small datasets, such as category id -> category name mappings, which will
not change during the course of indexing.
2. Queries which are heavy on db resources per-query, particularly if the query for an individual
record is slow, and can't be fixed easily on the db side for whatever reason.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message