Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of erickerickson@gmail.com
 designates 209.85.216.176 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=YKMc4g9eAt4hFeNkHubCxqPZNXkoIGJSrykVeLjSwp45dMrfh4YIm9J4OFimDo1mJD
         aWQU76NHC8S8EAIUcGhMQ6zU1GmzKI6cTlwXKcEbBKxTx5Fe+NvL8S4hkrNTV7wsroF5
         0xS+G8/RHlkHf5kz/38YLM66Aj6nfvdMe4Uqg=
MIME-Version: 1.0
In-Reply-To: <AANLkTikSCCWD7R7js9Rw4m9TKcNcA0Bk7Nim5DvbipWd@mail.gmail.com>
References: <AANLkTin27g2dwfP4GBKKdfAHMBwzqQU_w80J+U+R+JBu@mail.gmail.com>
	<001901cb5d34$92cef3b0$b86cdb10$@thetaphi.de>
	<AANLkTintcq21ULqswpvJuwBiegUGNWcci_m1gm+PQwcz@mail.gmail.com>
	<004301cb5d9e$91a5cfd0$b4f16f70$@thetaphi.de>
	<AANLkTim7qCKop6=P1n=Cq=ZMWUbuoJy5aGMZ3bfExYX9@mail.gmail.com>
	<AANLkTimpjB-SNwx9ZU+Ow=uDf+eCpUuG+2KE-RzbR0op@mail.gmail.com>
	<AANLkTikJqgyv3zZ5GSLZHrVLDA9XXyd1Ah98LZ-W9Ovc@mail.gmail.com>
	<AANLkTimSJYn=UhybgxpT5uBhVBekmEdSM8N+PLOJLJqq@mail.gmail.com>
	<AANLkTikSCCWD7R7js9Rw4m9TKcNcA0Bk7Nim5DvbipWd@mail.gmail.com>
Date: Mon, 11 Oct 2010 16:45:20 -0400
Message-ID: <AANLkTi=gvrq7=3gQ-b4HxK8rbE1EDoeGK2w3w-zZ=vjW@mail.gmail.com>
Subject: Re: flushing index
From: Erick Erickson <erickerickson@gmail.com>
To: java-user@lucene.apache.org
Content-Type: multipart/alternative; boundary=0016362841a450d82b04925d70a3

--0016362841a450d82b04925d70a3
Content-Type: text/plain; charset=ISO-8859-1

It's a good idea to start a new thread when asking a different question,
see:
http://people.apache.org/~hossman/#threadhijack

<http://people.apache.org/~hossman/#threadhijack>I have to ask why you want
to integrate the RAM directory. If you're using it
to speed up indexing, you're probably making way more work for yourself
than you need to. If you're trying to do something with Near Real Time, one
suggestion is to just not bother. Add docs to the RAM directory AND your
FSDirectory simultaneously. The data you index to FSDir won't be visible
until you reopen the FSDir reader, so your flush could then be just
reopen everything...

Best
Erick

On Mon, Oct 11, 2010 at 1:34 PM, Yakob <jacobian@opensuse-id.org> wrote:

> On 9/29/10, Erick Erickson <erickerickson@gmail.com> wrote:
> > Nope, never used jNotify, so I don't have any code handy...
> >
> > Good luck!
> > Erick
> >
>
> so I did try JNotify but there is seems to be some bug in it that I
> find it hards to integrate in my lucene source code.so I had to try a
> looping option instead.
>
>
> http://stackoverflow.com/questions/3840844/error-exception-access-violation-in-jnotify
>
> so anyway, I had another question now. I was trying to make a lucene
> source code that can do indexing and store them first in a memory
> using RAMDirectory and then flush this index in a memory into a disk
> using FSDirectory. I had done some modifications of this code but to
> no avail. maybe some of you can help me out a bit.
> here is the source code again.
>
> import java.io.File;
> import java.io.FileReader;
> import java.io.IOException;
> import org.apache.lucene.analysis.SimpleAnalyzer;
> import org.apache.lucene.document.Document;
> import org.apache.lucene.document.Field;
> import org.apache.lucene.index.IndexWriter;
> import org.apache.lucene.store.FSDirectory;
>
>
> public class SimpleFileIndexer {
>
>
>        public static void main(String[] args) throws Exception   {
>
>                 int i=0;
>                while(i<10) {
>                 File indexDir = new
> File("C:/Users/Raden/Documents/lucene/LuceneHibernate/adi");
>                File dataDir = new
> File("C:/Users/Raden/Documents/lucene/LuceneHibernate/adi");
>                String suffix = "txt";
>
>                SimpleFileIndexer indexer = new SimpleFileIndexer();
>
>                int numIndex = indexer.index(indexDir, dataDir, suffix);
>
>                System.out.println("Total files indexed " + numIndex);
>                 i++;
>                Thread.sleep(10000);
>
>                }
>        }
>
>
>
>        private int index(File indexDir, File dataDir, String suffix) throws
> Exception {
>
>                IndexWriter indexWriter = new IndexWriter(
>                                FSDirectory.open(indexDir),
>                                new SimpleAnalyzer(),
>                                true,
>                                IndexWriter.MaxFieldLength.LIMITED);
>                indexWriter.setUseCompoundFile(false);
>
>                indexDirectory(indexWriter, dataDir, suffix);
>
>                int numIndexed = indexWriter.maxDoc();
>                indexWriter.optimize();
>                indexWriter.close();
>
>                return numIndexed;
>
>        }
>
>        private void indexDirectory(IndexWriter indexWriter, File dataDir,
> String suffix) throws IOException {
>                File[] files = dataDir.listFiles();
>                for (int i = 0; i < files.length; i++) {
>                        File f = files[i];
>                        if (f.isDirectory()) {
>                                indexDirectory(indexWriter, f, suffix);
>                        }
>                        else {
>                                indexFileWithIndexWriter(indexWriter, f,
> suffix);
>                        }
>                }
>        }
>
>        private void indexFileWithIndexWriter(IndexWriter indexWriter, File
> f, String suffix) throws IOException {
>                if (f.isHidden() || f.isDirectory() || !f.canRead() ||
> !f.exists()) {
>                        return;
>                }
>                if (suffix!=null && !f.getName().endsWith(suffix)) {
>                        return;
>                }
>                System.out.println("Indexing file " + f.getCanonicalPath());
>
>                Document doc = new Document();
>                doc.add(new Field("contents", new FileReader(f)));
>                doc.add(new Field("filename", f.getCanonicalPath(),
> Field.Store.YES,
> Field.Index.ANALYZED));
>
>                indexWriter.addDocument(doc);
>        }
>
> }
>
> so what's the best way for me to integrate RAMDirectory in that source
> code before putting them in FSDirectory. any help would be appreciated
> though.
> thanks
>
>
> --
> http://jacobian.web.id
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

--0016362841a450d82b04925d70a3--