lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ingolf Tobias Rothe" <ir268...@inf.tu-dresden.de>
Subject Re: index and access to lines of a CSV file
Date Fri, 14 Dec 2007 10:43:19 GMT
Hello Mike,

thank you for the answer.  Currently I hold this datastructure in a
HashTable in Memory but the ressource consumtion is very high.
Lucene looks easy in the way of usage and is suposed to be extremely
perfomant.  I though also to use later the abilities to lucene to attach
parameters to search results so I do not neccessarily need to process the
record.

I checked berkley db but it is not free for comercial use. What would be a
implementation for btree?

Thank you,

Tobias


> On 13-Dec-07, at 3:26 PM, Tobias Rothe wrote:
>
>> I got a quick question.  I am handling hughe CSV files. They start
>> with a key in the first column and are followed by data.
>> I need to retrieve randomly this data based on the key.  So it is
>> kind of a search where I give a unique key and ideally access to
>> the right line.
>> The file contains about 200.000 lines or more.  I am not sure if
>> Lucene can handle things like that and I did not really find a hint
>> on this  topic.  So I hope to find help here.
>
> Is that all you're doing?  You could accomplish the above with Lucene
> but it isn't really needed for that.  You need some kind of on-disk
> key->value mapper.  Something like a berkeley db hashtable or btree
> should work (store each line as a key/value pair).
>
> -Mike
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message