lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <>
Subject Re: [jira] Commented: (SOLR-567) SolrCore Pluggable
Date Fri, 05 Sep 2008 14:29:54 GMT
These all sound like good ideas for solr2. The ability to handle changes 
on the fly easily across many machines would be an awesome place to 
reach. Dynamically changing field schema/stopword stuff is also a great 
feature to have.

I think the key to reaching those goals is to really join the solr open 
source process and push at it a piece at a time. I think almost all of 
the features you have talked about make sense for the future of solr in 
one way or another. Take advantage of solrs popularity and I think 
eventually we can have more people helping make these goals a reality. 
It might not happen as fast as you'd like (already having all this Ocean 
code), but I think the long run returns will be much higher.

You already have a great start with the patches and wiki info, but I 
think I am suggesting leaving the Ocean part out a little more, and 
maybe tackling these issues more form a Lucene/Solr perspective, using 
the Ocean ideas and code as leverage to move forward faster.

- Mark

Jason Rutherglen wrote:
> That may be the way to go, however there are many issues that you will
> probably run into, the same ones I ran into that made the integration
> difficult as mentioned.  I could be wrong however.  Most "realtime
> search" is something on the order of a 5-10 second delay and there is
> no transaction log.  I guess I made the goal of Ocean to be a
> replacement for SQL databases, 0 second delay, transaction log
> reliability, easy scalability, easy to add heavily customized queries
> (span, payload, custom similarities, etc), and maximum uptime.  I
> wanted to concentrate a lot on the grid computing part and leave the
> rest of system as generic as possible so that what I consider to be
> application specific code is not part of the search server.  For
> example I consider custom Analyzers to be application specific code.
> I want those dynamically loaded so I don't have to reboot 100 servers
> if I make a change to one.  Add a stop word?  Well then don't have to
> reboot 100 servers.  Synonyms?  The list does on and on.  Search is
> fluid and dynamic.
> On Fri, Sep 5, 2008 at 10:07 AM, Noble Paul നോബിള്‍ नोब्ळ्
> <> wrote:
>> On Fri, Sep 5, 2008 at 7:23 PM, Mark Miller <> wrote:
>>> IMO, you are underestimating the difficulty of integrating Ocean with Solr's
>>> current API's.
>> OK. you are right. Actually ,I did not mean the ocean integration. I
>> am mostly interested in the Realtime search part.  If we take one baby
>> step at a time it may be easy for us .  We can add features one by
>> one, but why don't we start with realtime search? (this sounds like an
>> immediately useful feature to an average solr user).
>>> Also, Jason has already mentioned that Ocean is much more than just realtime
>>> search. Adding realtime search to something like solr 1.5 is a different
>>> goal than possibly integrating the Ocean work that has been done / is
>>> planned, which seems like a very large scope project and if done would
>>> certainly seem to merit a 2.0 change in its own right.
>>> Still seems large and nebulous to me at the moment...just like solr 2. They
>>> go well together in my mind <g>
>>> Noble Paul നോബിള്‍ नोब्ळ् wrote:
>>>> Postponing Ocean Integration towards 2.0 is not a good idea. First of
>>>> all we do not know when 2.0 is going to happen. delaying  such a good
>>>> feature till 2.0 is wasting time.
>>>> My assumption was that Actually realtime search may have nothing to do
>>>> with the core itself . It may be fine with a Pluggable
>>>> SolrIndexSearcherFactory/SolrIndexWriterFactory . Ocean can have a
>>>> unified reader-writer which may choose to implement both in one class.
>>>> A total rewrite has its own problems. Achieving consensus on how
>>>> things should change is time consuming. So it will keep getting
>>>> delayed.  If with a few changes we can start the integration, that is
>>>> the best way forward . Eventually , we can slowly ,  evolve to a
>>>> better design. But, the design need not be as important as the feature
>>>> itself.
>>>> On Fri, Sep 5, 2008 at 6:46 PM, Yonik Seeley <> wrote:
>>>>> On Fri, Sep 5, 2008 at 9:03 AM, Jason Rutherglen
>>>>> <> wrote:
>>>>>> Ok, SOLR 2 can be a from the ground up rewrite?
>>>>> Sort-of... I think that's up for discussion at this point, but enough
>>>>> should change that keeping Java APIs back compatible is not a priority
>>>>> (just my opinion of course).  Supporting the current main search and
>>>>> update interfaces and migrating most of the handlers shouldn't be that
>>>>> difficult.  We should be able to provide relatively painless back
>>>>> compatibility for the 95% of Solr users that don't do any custom
>>>>> Java.... and the others hopefully won't mind migrating their stuff to
>>>>> get the cool new features :-)
>>>>> As far as SolrCore goes... I agree it's probably best to not do
>>>>> pluggability at that level.
>>>>> The way that Lucene has evolved, and may evolve (and how we want Solr
>>>>> to evolve), it seems like we want more of a combo
>>>>> IndexReader/IndexWriter interface.  It also needs (optional)
>>>>> optimistic concurrency... that was also assumed in the discussions
>>>>> about bailey.
>>>>> -Yonik
>> --
>> --Noble Paul

View raw message