lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yakob <>
Subject problem with incremental update in lucene
Date Wed, 01 Dec 2010 05:23:34 GMT
I am creating a program that can index many text files in different
folder. so that's mean every folder that has text files get indexed
and its index are stored in another folder. so this another folder
acts like a universal index of all files in my computer. and I am
using lucene to achieve this because lucene fully supported
incremental update. this is the source code into which I use it for

public class SimpleFileIndexer {

public static void main(String[] args) throws Exception   {

    int i=0;
    while(i<2) {
    File indexDir = new File("C:/Users/Raden/Documents/myindex");
    File dataDir = new File("C:/Users/Raden/Documents/indexthis");
    String suffix = "txt";

    SimpleFileIndexer indexer = new SimpleFileIndexer();

    int numIndex = indexer.index(indexDir, dataDir, suffix);

    System.out.println("Total files indexed " + numIndex);


private int index(File indexDir, File dataDir, String suffix) throws Exception {
    RAMDirectory ramDir = new RAMDirectory();          // 1
    IndexWriter indexWriter = new IndexWriter(
            ramDir,                                    // 2
            new StandardAnalyzer(Version.LUCENE_CURRENT),
    indexDirectory(indexWriter, dataDir, suffix);
    int numIndexed = indexWriter.maxDoc();

    Directory.copy(ramDir,, false); // 3

    return numIndexed;

private void indexDirectory(IndexWriter indexWriter, File dataDir,
String suffix)  throws IOException {
    File[] files = dataDir.listFiles();
    for (int i = 0; i < files.length; i++) {
        File f = files[i];
        if (f.isDirectory()) {
            indexDirectory(indexWriter, f, suffix);
        else {
            indexFileWithIndexWriter(indexWriter, f, suffix);

private void indexFileWithIndexWriter(IndexWriter indexWriter, File f,
String suffix) throws IOException {
    if (f.isHidden() || f.isDirectory() || !f.canRead() || !f.exists()) {
    if (suffix!=null && !f.getName().endsWith(suffix)) {
    System.out.println("Indexing file " + f.getCanonicalPath());

    Document doc = new Document();
    doc.add(new Field("contents", new FileReader(f)));
doc.add(new Field("filename", f.getCanonicalPath(), Field.Store.YES,
} }

the problem I am having now is that the indexing program I created
above seem can't do any incremental update. I mean I can search for a
text file but only for the file that existed in the last folder to
which I already indexed, and the other previous folder that I had
already indexed seems to be missing in the search result and didn't
get displayed. can you tell me what went wrong in my code? I just
wanted to be able to have incremental update feature in my source
code. so in essence, my program seems to be overwriting the existing
index with the new one instead of merging it.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message