lucy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marvin Humphrey <>
Subject Re: [lucy-dev] Improving the ClusterSearcher
Date Wed, 28 Dec 2011 19:43:08 GMT
On Tue, Dec 27, 2011 at 11:22:57PM +0100, Nick Wellnhofer wrote:
>> I think this problem is solved if we move the code establishing the listening
>> socket from SearchServer#new to the top of SearchServer#serve.
> Yes, that should work. It's not ideal but maybe it's the best  
> compromise. I'll give it a try.

What aspects continue to trouble you?

I'm quite happy with the refinements we've made. :)  By exposing
handle_request(), we fix an important problem with SearchServer: decisions
about forking/pre-forking etc. really belong in the realm of the application,
not the library.  Now it will be possible for application developers to make
their own call without monkey-patching

Even better, that improvement has been achieved without exposing any
implementation details of the protocol used by LucyX::Remote classes to encode
messages sent over socket connections.  We are free to evolve our
serialization techniques along the lines of what was suggested on this list a
few weeks ago without concern for breaking public APIs.

I can think of a couple minor tweaks I'd like to see.  It would be nice to
move the "port" argument from SearchServer#new to SearchServer#serve, and it
would be nice to do away with "password" entirely for all LucyX::Remote

It would also be nice if SearchServer#serve had better out-of-the-box
performance degradation characteristics under increasing load, but I don't
think that's a requirement.  We can offer a clear path forward by documenting
how to roll-your-own solution with handle_request().

There are lots of problems yet to be solved with regards to using Lucy in a
clustered environment -- we haven't tackled indexing or updates at all -- but
I think we're making excellent forward progress while avoiding painting
ourselves into a corner.  Thanks for taking this on!

Marvin Humphrey

View raw message