lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bryan LaPlante" <>
Subject Indexing a database
Date Tue, 10 Jun 2003 16:09:23 GMT

I need some input about what direction to take. I have written a package for
indexing a database using a query or list of tables to be indexed. I wanted
control over how each column in each row of the result gets indexed or not.

The structure first and then the problem where advice is needed.

    //create an instance of :
    ds = DataStore(String driver,String uri,String pswd, String user)
    // pass ds to:
    dir = DSDirectory(DataStore ds,String query);
Calling dir.list() now will produce the entire resultset made up of a
DSFile() representing a row in the set and a Hashtable of attributes stored
internally to the dsfile object representing each column.

The problem:
    If you didn't guess before, this is quite memory intensive when you are
talking about a sizable recordset. I need a way to either hold a reference
to the records and let the user incrementally request the next n number of
rows to index or I need to store the records in a temporary location
(non-memory) where they can be retrieved on request.

Possible solution:
    I have one idea to create a temp Lucene index representing the rows and
columns and then when the list method is called I could retrieve the data
from the index there by not have the in memory constraints.


Bryan LaPlante

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message