lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yakob <jacob...@opensuse-id.org>
Subject Re: flushing index
Date Mon, 11 Oct 2010 17:34:48 GMT
On 9/29/10, Erick Erickson <erickerickson@gmail.com> wrote:
> Nope, never used jNotify, so I don't have any code handy...
>
> Good luck!
> Erick
>

so I did try JNotify but there is seems to be some bug in it that I
find it hards to integrate in my lucene source code.so I had to try a
looping option instead.

http://stackoverflow.com/questions/3840844/error-exception-access-violation-in-jnotify

so anyway, I had another question now. I was trying to make a lucene
source code that can do indexing and store them first in a memory
using RAMDirectory and then flush this index in a memory into a disk
using FSDirectory. I had done some modifications of this code but to
no avail. maybe some of you can help me out a bit.
here is the source code again.

import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import org.apache.lucene.analysis.SimpleAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.store.FSDirectory;


public class SimpleFileIndexer {
	
	
	public static void main(String[] args) throws Exception   {
		
		int i=0;
		while(i<10) {
		File indexDir = new
File("C:/Users/Raden/Documents/lucene/LuceneHibernate/adi");
		File dataDir = new
File("C:/Users/Raden/Documents/lucene/LuceneHibernate/adi");
		String suffix = "txt";
		
		SimpleFileIndexer indexer = new SimpleFileIndexer();
		
		int numIndex = indexer.index(indexDir, dataDir, suffix);
		
		System.out.println("Total files indexed " + numIndex);
		i++;
		Thread.sleep(10000);
		
		}
	}

	
	
	private int index(File indexDir, File dataDir, String suffix) throws
Exception {
		
		IndexWriter indexWriter = new IndexWriter(
				FSDirectory.open(indexDir),
				new SimpleAnalyzer(),
				true,
				IndexWriter.MaxFieldLength.LIMITED);
		indexWriter.setUseCompoundFile(false);
		
		indexDirectory(indexWriter, dataDir, suffix);
		
		int numIndexed = indexWriter.maxDoc();
		indexWriter.optimize();
		indexWriter.close();
		
		return numIndexed;
		
	}
	
	private void indexDirectory(IndexWriter indexWriter, File dataDir,
String suffix) throws IOException {
		File[] files = dataDir.listFiles();
		for (int i = 0; i < files.length; i++) {
			File f = files[i];
			if (f.isDirectory()) {
				indexDirectory(indexWriter, f, suffix);
			}
			else {
				indexFileWithIndexWriter(indexWriter, f, suffix);
			}
		}
	}
	
	private void indexFileWithIndexWriter(IndexWriter indexWriter, File
f, String suffix) throws IOException {
		if (f.isHidden() || f.isDirectory() || !f.canRead() || !f.exists()) {
			return;
		}
		if (suffix!=null && !f.getName().endsWith(suffix)) {
			return;
		}
		System.out.println("Indexing file " + f.getCanonicalPath());
		
		Document doc = new Document();
		doc.add(new Field("contents", new FileReader(f)));		
		doc.add(new Field("filename", f.getCanonicalPath(), Field.Store.YES,
Field.Index.ANALYZED));
		
		indexWriter.addDocument(doc);
	}
	
}

so what's the best way for me to integrate RAMDirectory in that source
code before putting them in FSDirectory. any help would be appreciated
though.
thanks


-- 
http://jacobian.web.id

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message