Mailing-List: contact lucy-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: lucy-dev@incubator.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Date: Mon, 4 Jul 2011 11:05:14 -0700
From: Marvin Humphrey <marvin@rectangular.com>
To: lucy-dev@incubator.apache.org
Message-ID: <20110704180514.GA10459@rectangular.com>
References: <BANLkTikbXe8DCzHqt=2c67LvEM7Wo4dbkg@mail.gmail.com>
 <20110701205259.GA22872@rectangular.com>
 <CABTa2P6kBKHmxUBfofTP4OotgxLz=4ckP98ehkpEMpRdQCeGjw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: 
 <CABTa2P6kBKHmxUBfofTP4OotgxLz=4ckP98ehkpEMpRdQCeGjw@mail.gmail.com>
User-Agent: Mutt/1.5.18 (2008-05-17)
Subject: Re: [lucy-dev] building withOUT perl

On Mon, Jul 04, 2011 at 10:33:15AM +0200, Torsten Curdt wrote:
> Hm. So there would be a dedicated release dist per language binding.

Not necessarily.

There has to be a dedicated "downstream" release for each *packaging system*
we might target -- such as CPAN, rubygems.org or PyPI -- because each insists
on certain conventions regarding directory layout and metadata.

I don't see why we would need a dedicated release dist for any language
without an associated packaging system, though.  You can just build from the
canonical Apache release artifacts, which is equivalent to building from a
checkout of svn trunk.  That option is of course available for any supported
binding language, regardless of the presence or absence of supplementary
downstream dists.

> >  IMO the Clownfish approach offers some nice advantages, not
> > �least the ability to override C classes in the native host language.
> 
> Interesting ...is there more?

Yes: named parameters, default argument values, transparent translation of
string/num/hash/array types and complex data structures, and a certain amount
of argument validation.

Method dispatch for Clownfish classes implemented in C (e.g. everything in the
Lucy core) is also much faster than what you get in a typical dynamic
language.  That's because it's a vtable-based double-pointer-dereference --
similar to C++ or Java -- rather than hash-based run-time method resolution
keyed off the method name a la Perl, Ruby, Python, etc.

> > These days, the majority of our development energies are going towards
> > expanding Lucy's availability (as opposed to say, improving search-related
> > features). �The first step was getting all the licensing issues resolved and
> > getting our inaugural release out.
> 
> How far is that process?

0.1.0 was released in June.

We still have two CPAN module dependencies which must be eliminated prior to
graduating from incubation (see <issues.apache.org/jira/browse/LEGAL-86>), but
they do not constrain our ability to make incubating releases.  Aside from
that, we are in the clear.

> > We're not quite at a place where adding a new binding is a turnkey process.
> > Once our next release is done, I'm going to go back to working on
> > <https://issues.apache.org/jira/browse/LUCY-142>, "Port Clownfish compiler to
> > C". �We're whittling down the barriers, inch by inch.
> 
> Uff ... that sounds like quite a task.

A lot of it is done.  It's also something I'm really enjoying working on.  :)
And rewriting the parser to use the Lemon parser generator rather than the
CPAN module Parse::RecDescent solves half of our remaining dependency
licensing issues.

> > If you want to wait until it's more straightforward to bind Lucy to Objective
> > C, feel free to lurk as some of our Ruby and TCL volunteers are doing right
> > now.
> 
> /me also enters lurk mode

Heh, glad to have you around.

> > �If you'd like to help us move along that path more quickly, as Joe
> > Shaefer has, we'll be happy to match you up with the tasks you'd find most
> > interesting to work on.
> 
> I have looked at "example-lang" but implementing the "Object" seems to
> require quite some low-level foo for every language impl.

Yes, that is true.  As Nate Kurz put it (IIRC), it is analogous to adding a
language to SWIG, rather than adding a SWIG binding.

> Is that even feasible in every language?

It is definitely feasible with Perl, Ruby, Python and Java.  The layer that is
now Clownfish was conceived by Dave Balmain (author of Ferret, a Ruby/C port
of Lucene) and myself for the original incarnation of Lucy as a Lucene
subproject, and those were the languages we had in mind.  (Dave subsequently
became inactive, which is why the current implementation was worked out for
Perl alone.)  Theoretically, Clownfish should work with any language which
supports single-inheritance class-based OOP.

> The perl/xs is the one and only "example-lang" implementation at this
> stage IIUC?

Correct.  There's now some code in trunk/ruby as well -- but it's in its
earliest stages.

Perhaps follow along while the Ruby bindings get implemented over the next few
months.  Aside from Dave Balmain, the people who grok Clownfish and understand
the rationale behind it the best appear to be those who have a significant
amount of experience with Perl's C API, such as Peter, Joe and Nate.  I hope
that once Clownfish is more cleanly separated from Perl and from Lucy's search
functionality that others without such expertise will find it more
approachable.

Marvin Humphrey