Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of sarowe@gmail.com designates
 209.85.213.178 as permitted sender)
Content-Type: text/plain; charset=windows-1252
Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\))
Subject: Re: Managed Schema
From: Steve Rowe <sarowe@gmail.com>
In-Reply-To: 
 <CADRon30e8ParDcmVxoNa6iE-ddiTji-Y2MVAEit3vo=tDHhqsg@mail.gmail.com>
Date: Fri, 15 Aug 2014 12:16:43 -0400
Content-Transfer-Encoding: quoted-printable
Message-Id: <3B5E1E02-E11D-472E-A2AB-AD0932F6B5B6@gmail.com>
References: 
 <CADRon31ZVsgasC3iEnwmcCdNnVyev=cq9W0YZQg1rjMtawgrrg@mail.gmail.com>
 <79EDEDBD-E157-4EE4-B35A-2B16D2623DF5@gmail.com>
 <CADRon30e8ParDcmVxoNa6iE-ddiTji-Y2MVAEit3vo=tDHhqsg@mail.gmail.com>
To: solr-user@lucene.apache.org

In the Lucene/Solr 4.X line, the average minor release interval has been =
10 weeks.  4.9 was released 7 weeks ago.  It takes 2-3 weeks to go from =
starting the release process to producing and announcing a release, so I =
expect someone will volunteer to perform the 4.10 release within the =
next week or three.

I don=92t think directly using the schema API will necessarily fix the =
issues you=92re seeing.

Tim Potter is working on SOLR-6249, which will likely improve your =
experience with managed schema/schemaless: =
<https://issues.apache.org/jira/browse/SOLR-6249>.  One thing he=92s =
planning to put in place is a way to query each replica's schema =
version, so instead of blindly waiting a fixed amount of time, you could =
poll all replicas until your changes are known to have been =
incorporated.  (Yes, I realize that forcing clients to manage =
distributed schema versioning is not ideal - I expect there will be =
server-side facilities to do something equivalent.)=20

Steve
www.lucidworks.com

On Aug 15, 2014, at 11:49 AM, Joseph Obernberger =
<joseph.obernberger@gmail.com> wrote:

> Thank you!  Any ideas when 4.10 will be released?
> I tried sleeping when adding fields, and that improved things, but =
since
> I'm in schema-less mode, sometimes that doesn't work either.  If I =
managed
> the schema all 'myself' using the schema API; would that work?
>=20
>=20
> On Fri, Aug 15, 2014 at 11:34 AM, Steve Rowe <sarowe@gmail.com> wrote:
>=20
>> Hi Joseph,
>>=20
>> SOLR-6137 <https://issues.apache.org/jira/browse/SOLR-6137> fixed the =
NPE
>> you encountered with 4.9, as well a couple other managed schema =
concurrency
>> issues - it will be included in the next release of Solr: 4.10.
>>=20
>> Steve
>>=20
>> On Aug 15, 2014, at 11:00 AM, Joseph Obernberger <
>> joseph.obernberger@gmail.com> wrote:
>>=20
>>> Hi - I've been using Solr Cloud in schema-less mode and am having =
some
>>> issues with 4.8.1 and 4.9.0 when adding lots of new fields.  In =
4.8.1
>> I'll
>>> get continuous messages that say:
>>> 134567307 [qtp968427990-2492] INFO  =
org.apache.solr.schema.IndexSchema  =E2
>>> Failed to persist managed schema at /configs/M1/managed-schema - =
version
>>> mismatch
>>> 134567315 [qtp968427990-2492] INFO  =
org.apache.solr.schema.IndexSchema  =E2
>>> Failed to persist managed schema at /configs/M1/managed-schema - =
version
>>> mismatch
>>> 134567323 [qtp968427990-2492] INFO  =
org.apache.solr.schema.IndexSchema  =E2
>>> Failed to persist managed schema at /configs/M1/managed-schema - =
version
>>> mismatch
>>>=20
>>> In 4.9.0, something similar happens, but it comes with a stack =
trace:
>>>=20
>>> 710115 [qtp851206054-45] ERROR org.apache.solr.schema.IndexSchema  =E2=

>> Error
>>> persisting managed schema at /configs/M1/managed-schema
>>> java.lang.NullPointerException
>>>       at
>>>=20
>> =
org.apache.solr.schema.IndexSchema.getCopyFieldProperties(IndexSchema.java=
:1418)
>>>       at
>>>=20
>> =
org.apache.solr.schema.IndexSchema.getNamedPropertyValues(IndexSchema.java=
:1375)
>>>       at
>> org.apache.solr.schema.IndexSchema.persist(IndexSchema.java:379)
>>>       at
>>>=20
>> =
org.apache.solr.schema.ManagedIndexSchema.persistManagedSchemaToZooKeeper(=
ManagedIndexSchema.java:137)
>>>       at
>>>=20
>> =
org.apache.solr.schema.ManagedIndexSchema.persistManagedSchema(ManagedInde=
xSchema.java:83)
>>>       at
>>>=20
>> =
org.apache.solr.schema.ManagedIndexSchema.addFields(ManagedIndexSchema.jav=
a:244)
>>>       at
>>>=20
>> =
org.apache.solr.schema.ManagedIndexSchema.addFields(ManagedIndexSchema.jav=
a:201)
>>>       at
>>>=20
>> =
org.apache.solr.schema.ManagedIndexSchema.addFields(ManagedIndexSchema.jav=
a:50)
>>>       at
>>>=20
>> =
org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory$Add=
SchemaFieldsUpdateProcessor.processAdd(AddSchemaFieldsUpdateProcessorFacto=
ry.java:318)
>>>       at
>>>=20
>> =
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateR=
equestProcessor.java:51)
>>>       at
>>>=20
>> =
org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(F=
ieldMutatingUpdateProcessor.java:117)
>>>       at
>>>=20
>> =
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateR=
equestProcessor.java:51)
>>>       at
>>>=20
>> =
org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(F=
ieldMutatingUpdateProcessor.java:117)
>>>       at
>>>=20
>> =
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateR=
equestProcessor.java:51)
>>>       at
>>>=20
>> =
org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(F=
ieldMutatingUpdateProcessor.java:117)
>>>       at
>>>=20
>> =
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateR=
equestProcessor.java:51)
>>>       at
>>>=20
>> =
org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(F=
ieldMutatingUpdateProcessor.java:117)
>>>       at
>>>=20
>> =
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateR=
equestProcessor.java:51)
>>>       at
>>>=20
>> =
org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(F=
ieldMutatingUpdateProcessor.java:117)
>>>       at
>>>=20
>> =
org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:9=
6)
>>>       at
>>>=20
>> =
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuter=
MostDocIterator(JavaBinUpdateRequestCodec.java:166)
>>>       at
>>>=20
>> =
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readItera=
tor(JavaBinUpdateRequestCodec.java:136)
>>>       at
>>> =
org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:225)
>>>       at
>>>=20
>> =
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamed=
List(JavaBinUpdateRequestCodec.java:121)
>>>       at
>>> =
org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:190)
>>>       at
>>> =
org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:116)
>>>       at
>>>=20
>> =
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(J=
avaBinUpdateRequestCodec.java:173)
>>>       at
>>>=20
>> =
org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoade=
r.java:106)
>>>       at
>>> =
org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:58)
>>>       at
>>>=20
>> =
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.j=
ava:92)
>>>       at
>>>=20
>> =
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Content=
StreamHandlerBase.java:74)
>>>       at
>>>=20
>> =
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBas=
e.java:135)
>>>       at org.apache.solr.core.SolrCore.execute(SolrCore.java:1962)
>>>       at
>>>=20
>> =
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java=
:777)
>>>       at
>>>=20
>> =
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.jav=
a:418)
>>>       at
>>>=20
>> =
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.jav=
a:207)
>>>       at
>>>=20
>> =
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandl=
er.java:1419)
>>>       at
>>>=20
>> =
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)=

>>>       at
>>>=20
>> =
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:1=
37)
>>>       at
>>>=20
>> =
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557=
)
>>>       at
>>>=20
>> =
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.ja=
va:231)
>>>       at
>>>=20
>> =
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.ja=
va:1075)
>>>       at
>>> =
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
>>>       at
>>>=20
>> =
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.jav=
a:193)
>>>       at
>>>=20
>> =
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.jav=
a:1009)
>>>       at
>>>=20
>> =
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:1=
35)
>>>       at
>>>=20
>> =
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHa=
ndlerCollection.java:255)
>>>       at
>>>=20
>> =
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollectio=
n.java:154)
>>>       at
>>>=20
>> =
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java=
:116)
>>>       at org.eclipse.jetty.server.Server.handle(Server.java:368)
>>>       at
>>>=20
>> =
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttp=
Connection.java:489)
>>>       at
>>>=20
>> =
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttp=
Connection.java:53)
>>>       at
>>>=20
>> =
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnec=
tion.java:953)
>>>       at
>>>=20
>> =
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(Abs=
tractHttpConnection.java:1014)
>>>       at
>> org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:953)
>>>      at
>>> =
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
>>>       at
>>>=20
>> =
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnect=
ion.java:72)
>>>       at
>>>=20
>> =
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketC=
onnector.java:264)
>>>       at
>>>=20
>> =
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.jav=
a:608)
>>>       at
>>>=20
>> =
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java=
:543)
>>>       at java.lang.Thread.run(Thread.java:745)
>>>=20
>>> This happens when adding lots of new fields to a SolrInputDocument =
from
>>> multiple threads hitting the cluster (right now just 2 shards).
>>> Any ideas / work-arounds?  Thank you very much!
>>>=20
>>> -Joe Obernberger
>>=20
>>=20