incubator-lucy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Balmain" <>
Subject Re: Charmonizer
Date Thu, 19 Oct 2006 03:03:47 GMT
On 10/19/06, Marvin Humphrey <> wrote:
> On Oct 18, 2006, at 10:17 AM, David Balmain wrote:
> > Also, how are you thinking about passing parameters to the compiler to
> > support LFS? I'm trying to figure out how I can add this to Ferret. I
> > guess I'll probably have to generate the Makefile. What do you think?
> With the Perl modules Module::Build and ExtUtils::CBuilder, I don't
> seem to need a Makefile. :)  CBuilder handles the compilation and
> linking tasks, and Module::Build has an up_to_date() function with
> which I can implement compilation-only-upon-modification.

For Ruby I can use the make alternative rake. But I'm thinking about
Ferret at the moment.

> I can spec extra flags to CBuilder's compile() function if turns out
> to be necessary.  However, CBuilder, by default, passes the same set
> of flags that were used when compiling the Perl executable (which are
> archived, along with a zillion other settings from Perl's Configure
> script, in the Config module).  On a RedHat 9 box I have access to,
> those flags include -D_LARGEFILE_SOURCE and -D_FILE_OFFSET_BITS=64,
> and I'm assuming that other Perl installations where LFS isn't the OS
> default also spec flags rather than defining macros within individual
> source files.

Unfortunately these values are defined as macros in Ruby.

> So I don't think I need to worry about it.  But if I did, here's how
> the call would work:
>      my $more_flags  = maybe_add_largefile_flags();
>      my $object_file = $cbuilder->compile(
>          source               => $source_path,
>          extra_compiler_flags => $more_flags,
>      );
> If you look at my Build.PL script, you'll see that I'm retrieving the
> flags from the Config module and passing them to ./charmonize via the
> input file.
>      print $infile_fh qq|
>          <charm_outpath>$lucyconf_path</charm_outpath>
>          <charm_compiler>$Config{cc}</charm_compiler>
>          <charm_ccflags>$Config{ccflags}</charm_ccflags>
>      |;
> If $Config{ccflags} doesn't include the necessary LFS flags, then
> Charmonizer's LargeFiles module will fail to generate the FTell and
> FSeek symbols.  That means that if Perl itself was compiled without
> Large File Support (which has been the default since Perl 5.6,
> released last millenium), Lucy won't build.  But I think I might be
> ready to just say "No LFS? No Lucy."

Any reason the native language needs to support LFS? If all access to
the index files is through Lucy, it shouldn't matter right?

> (Stylistic footnote: those should be changed to Fseek and Ftell.  The
> alternate capitalization is a glitch, not worth explaining.)
> Does Ruby provide a module similar to Perl's Config?  If not, we
> might need to pursue another route and have Charmonizer #define LFS
> macros when needed.  However, I think that's a bad idea, because it
> can lead to the kind of conflict described in that article I linked
> to earlier in my "Large File Support" post. Requiring the flags is
> fail-safe, kicking the problem from the Charmonizer library up to the
> app; defining the LFS symbols within the Charmonizer library is fail-
> dangerous.

Ruby does have a Config module just like Perl. Unfortunately, as I
said earlier, the LFS flags are defined as macros in config.h so I
don't have access to them in Ruby.

One other thing. Have you thought about detecting dirent.h in
charmonizer? Are we going to need any directory reading functions in
Lucy? I use it to clear the directory when the IndexWriter create flag
is set to true but I guess this isn't really necessary.


View raw message