lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: madvise(ptr, len, MADV_SEQUENTIAL)
Date Fri, 19 Jun 2009 08:15:31 GMT
But then we also need to map, when writing to files, which is hard to do,
because you do not know how large the mapping area will be (unknown
filesize). As Earwin suggested, we could change MMapDirectory to also mmap
on writing, but maps the buffers in something we call "pages". Filesizes of
written files are then always multiples of this "page size", if this is not
a problem for lucene (e.g. making the random access file larger than
actually needed, having 0-bytes at the unused end). Maybe we need than an
api to tell the directory implementation, how large a segment may become
before writing (when merging, this should be possible, as the filesize is
about the sum of all merged segments).

 

On windows, we then will also rely on the cleaner()-hack to correctly close
files (see current MMap in lucene-trunk, where the cleaner-hack can
optionally be enabled, without guaranties).

 

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

  _____  

From: Jason Rutherglen [mailto:jason.rutherglen@gmail.com] 
Sent: Friday, June 19, 2009 12:33 AM
To: Alan Bateman
Cc: java-dev@lucene.apache.org; nio-discuss@openjdk.java.net
Subject: Re: madvise(ptr, len, MADV_SEQUENTIAL)

 

Hmm... So the list at the bottom of this page looks accurate?
http://www.gnu.org/software/hello/manual/gnulib/posix_005ffadvise.html

If it is, then posix_fadvise works on Linux only?  

Perhaps madvise will be better then (judging by the much smaller unsupported
list)?  It seems to run on most platforms:
http://www.gnu.org/software/hello/manual/gnulib/madvise.html

On Wed, Jun 17, 2009 at 2:19 AM, Alan Bateman <Alan.Bateman@sun.com> wrote:

Jason Rutherglen wrote:

Alan,

Do you think something like FileDescriptor.setAdvise (mirroring
posix_fadvise) makes sense?

-J

Something like a posix_fadvise would be more appropriate for FileChannel or
maybe as a usage hint when opening the file (the new APIs for opening files
are extensible to allow for additional options in the future or even
implementation specific options). I don't think we've had much interest in
doing this, maybe because it would be a no-op on many operating systems.

-Alan.

 


Mime
View raw message