Mailing-List: contact lucene-dev-help@jakarta.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Lucene Developers List" <lucene-dev@jakarta.apache.org>
Message-ID: <003601c32f6a$a8858f70$d9321c41@vincent>
Reply-To: "Bryan LaPlante" <blaplante@netwebapps.com>
From: "Bryan LaPlante" <blaplante@netwebapps.com>
To: <lucene-dev@jakarta.apache.org>
Subject: Indexing a database
Date: Tue, 10 Jun 2003 11:09:23 -0500
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

Hi,

I need some input about what direction to take. I have written a package for
indexing a database using a query or list of tables to be indexed. I wanted
control over how each column in each row of the result gets indexed or not.

The structure first and then the problem where advice is needed.

Structure:
    //create an instance of :
    ds = DataStore(String driver,String uri,String pswd, String user)
    // pass ds to:
    dir = DSDirectory(DataStore ds,String query);
Calling dir.list() now will produce the entire resultset made up of a
DSFile() representing a row in the set and a Hashtable of attributes stored
internally to the dsfile object representing each column.

The problem:
    If you didn't guess before, this is quite memory intensive when you are
talking about a sizable recordset. I need a way to either hold a reference
to the records and let the user incrementally request the next n number of
rows to index or I need to store the records in a temporary location
(non-memory) where they can be retrieved on request.

Possible solution:
    I have one idea to create a temp Lucene index representing the rows and
columns and then when the list method is called I could retrieve the data
from the index there by not have the in memory constraints.

Thoughts?

Bryan LaPlante


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org