lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gopikrishnan Subramani <gopi.subram...@gmail.com>
Subject Re: IO bandwidth throttling
Date Tue, 02 Aug 2005 20:27:35 GMT
Ok, let me rephrase the question. Assuming the RAMDirectory holds
approximately 500 MB of data which needs to be written to the
filesystem, I'm afraid that sending this much data in one shot might
choke the NFS. Is there a parameter with FSDirectory with which I can
instruct Lucene to restrict the amount of data my application writes
to the filesystem per second.

BTW, I could see maxMergeDocs, but not maxBufferedDocs with the
IndexWriter. Otis, is there really some parameter with that name? I
think maxMergeDocs may not work well here because the size of the
documents are pretty random, ranging from few hundred bytes to tens of
megabytes.

Thanks,
Gopi

On 8/3/05, Otis Gospodnetic <otis_gospodnetic@yahoo.com> wrote:
> While not exactly what you are describing, you can use one of the
> IndexWriter parameters (maxBufferedDocs) to control the size of the
> RAMDirectory that's used as a buffer during indexing.
> 
> Otis
> 
> --- Gopikrishnan Subramani <gopi.subramani@gmail.com> wrote:
> 
> > Hello,
> >
> > Is there a way I can control the IO bandwidth utilized by Lucene?
> >
> > Here is my scenario. RAMDirectory is used to build a in-memory index
> > and finally the index size approaches a limit, the contents are
> > flushed to a FSDirectory. The index size could be approximately 512
> > MB. I'm a bit concerned that this might affect other NFS clients of
> > the same NFS server, even if it's for a short period. To avoid this I
> > would like to implement some sort of limit on the amount of data
> > transferred to the filesystem akin to the --bwlimit with the rsync
> > command.
> >
> > Thanks,
> > Gopi
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message