lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantyn Smirnov <inject...@yahoo.com>
Subject Re: Index one huge text file
Date Mon, 25 Jul 2011 09:17:11 GMT
If you read your file as a stream, i.e. line-by-line without buffering it in
RAM, you shall have no problems with performance, as 60k lines is a piece of
cake :).

You can try using LineNumberReader:

Reader lnr = new LineNumberReader( new FileReader( new File(
'/path/to/your/file' ) ) )
String line
while( null != (line = lnr.readLine()) ){
// do stuff with line
}

30 min ago I wrote a script to read a file of 20k lines and sort its' lines
alphabetically, and it took like 1.5 sec.

--
View this message in context: http://lucene.472066.n3.nabble.com/Index-one-huge-text-file-tp3191605p3196951.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message