Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (herse.apache.org: domain of soeren.pekrul@gmx.de
 designates 213.165.64.20 as permitted sender)
Message-ID: <4581A328.2030909@gmx.de>
Date: Thu, 14 Dec 2006 20:16:56 +0100
From: Soeren Pekrul <soeren.pekrul@gmx.de>
User-Agent: Mozilla Thunderbird 1.0.7 (Windows/20050923)
MIME-Version: 1.0
To: java-user@lucene.apache.org
Subject: Re: Lucene & LSA
References: <7858202.post@talk.nabble.com> <4581153D.9060106@gmx.de>
 <7870561.post@talk.nabble.com>
In-Reply-To: <7870561.post@talk.nabble.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

mariolone wrote:
> They are successful to extract the matrix. 
> But with collections of large documents is not one too much expensive
> solution? 

I have a quite small collection with 14,960 documents and 29,828 unique 
terms. If I remember right it took a few minutes on a normal laptop 
computer to iterate the terms and documents. I stored the matrix in mySQL:

CREATE TABLE term_document_matrix (
	term VARCHAR( 32 ) NOT NULL ,
	document INT NOT NULL ,
	weight DOUBLE NOT NULL DEFAULT '0',
	PRIMARY KEY (term, document)
);

You can see it is not a real matrix just a normal table in the 
relational model. I stored the weights greater than 0 only, so I have 
much less entries than 14,960 x 29,828 = 446,226,880 (in my case 159,407).

> it is possible to extract the matrix from the indexing file? 

I don’t know any API to extract the matrix from the index file directly.

Sören

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org