Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of jason.rutherglen@gmail.com
 designates 64.233.184.230 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version
         :content-type:content-transfer-encoding:content-disposition
         :references;
        b=BfyzUvF/U4XnH71pKx5fuM5yTeoe5HF5ENVhtlYb0ULflbrmIHSBTvLFjWcTazm4++
         GZBF709aQQVM2OIDOyUed543Q/dQlXThqZ74uzRBTYZRyXUXdsVuQmVVO3dfhwa5YxdZ
         CRNWunC9EGCwZrydUXvQMk3yo214tLDtORxAs=
Message-ID: <85d3c3b60809180443k32b2dab2w5167c2140b549c3d@mail.gmail.com>
Date: Thu, 18 Sep 2008 07:43:47 -0400
From: "Jason Rutherglen" <jason.rutherglen@gmail.com>
To: solr-user@lucene.apache.org
Subject: Re: Some new SOLR features
In-Reply-To: <c68e39170809171121o65cefa4bxf6e95fb773f63dbb@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <85d3c3b60809150734m44797b14m7fae0ce963cf71c8@mail.gmail.com>
	 <FECDA0AA-48B2-4221-88F1-C21F73722E52@gmail.com>
	 <85d3c3b60809160712i24573a03g4a912e2e092fb667@mail.gmail.com>
	 <A4EF1B33-DB4A-45D4-BE83-F20B01A1EC23@gmail.com>
	 <19515526.post@talk.nabble.com>
	 <587CADDC-BBBD-4915-90D7-161D3F5A3474@gmail.com>
	 <19516242.post@talk.nabble.com>
	 <85d3c3b60809171027m2f453f10k5f20dbb82f003919@mail.gmail.com>
	 <c68e39170809171121o65cefa4bxf6e95fb773f63dbb@mail.gmail.com>

> multi-core allows you to instantiate a completely
> new core and swap it for the old one, but it's a bit of a heavyweight
> approach.

Multi core seems like more of a hack to get around running multiple
JVMs.  It doesn't seem like the most elegant solutions for most
problems because usually the same configuration files can be used, and
the schemas will be different.  This is true for the systems because
they are loading data from an SQL database for indexing which is in
separate tables.  I put a field in the documents demarking the
different tables or object types which then is filtered on.  This is
not ideal if one wants separation of the term frequencies between the
tables and possibly for speed if one table is really small, it should
return results faster and not still iterate over the data of the other
tables.

On Wed, Sep 17, 2008 at 2:21 PM, Yonik Seeley <yonik@apache.org> wrote:
> On Wed, Sep 17, 2008 at 1:27 PM, Jason Rutherglen
> <jason.rutherglen@gmail.com> wrote:
>> If the configuration code is going to be rewritten then I would like
>> to see the ability to dynamically update the configuration and schema
>> without needing to reboot the server.
>
> Exactly.  Actually, multi-core allows you to instantiate a completely
> new core and swap it for the old one, but it's a bit of a heavyweight
> approach.
>
> The key is finding the right granularity of change.
> My current thought is that a schema object would not be mutable, but
> that one could easily swap in a new schema object for an index at any
> time.  That would allow a single request to see a stable view of the
> schema, while preventing having to make every aspect of the schema
> thread-safe.
>
>> Also I would like the
>> configuration classes to just contain data and not have so many
>> methods that operate on the filesystem.
>
> That's the plan... completely separate the serialized and in memory
> representations.
>
>> This way the configuration
>> object can be serialized, and loaded by the server dynamically.  It
>> would be great for the schema to work the same way.
>
> Nothing will stop one from using java serialization for config
> persistence, however I am a fan of human readable for config files...
> so much easier to debug and support.  Right now, people can
> cut-n-paste relevant parts of their config in email for support, or to
> a wiki to explain things, etc.
>
> Of course, if you are talking about being able to have custom filters
> or analyzers (new classes that don't even exist on the server yet),
> then it does start to get interesting.  This intersects with
> deployment in general... and I'm not sure what the right answer is.
> What if Lucene or Solr needs an upgrade?  It would be nice if that
> could also automatically be handled in a a large cluster... what are
> the options for handling that?  Is there a role here for OSGi to play?
>  It sounds like at least some of that is outside of the Solr domain.
>
> An alternative to serializing everything would be to ship a new schema
> along with a new jar file containing the custom components.
>
> -Yonik
>