lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Rowe <sar...@gmail.com>
Subject Re: Managed Schema
Date Fri, 15 Aug 2014 16:16:43 GMT
In the Lucene/Solr 4.X line, the average minor release interval has been 10 weeks.  4.9 was
released 7 weeks ago.  It takes 2-3 weeks to go from starting the release process to producing
and announcing a release, so I expect someone will volunteer to perform the 4.10 release within
the next week or three.

I don’t think directly using the schema API will necessarily fix the issues you’re seeing.

Tim Potter is working on SOLR-6249, which will likely improve your experience with managed
schema/schemaless: <https://issues.apache.org/jira/browse/SOLR-6249>.  One thing he’s
planning to put in place is a way to query each replica's schema version, so instead of blindly
waiting a fixed amount of time, you could poll all replicas until your changes are known to
have been incorporated.  (Yes, I realize that forcing clients to manage distributed schema
versioning is not ideal - I expect there will be server-side facilities to do something equivalent.)


Steve
www.lucidworks.com

On Aug 15, 2014, at 11:49 AM, Joseph Obernberger <joseph.obernberger@gmail.com> wrote:

> Thank you!  Any ideas when 4.10 will be released?
> I tried sleeping when adding fields, and that improved things, but since
> I'm in schema-less mode, sometimes that doesn't work either.  If I managed
> the schema all 'myself' using the schema API; would that work?
> 
> 
> On Fri, Aug 15, 2014 at 11:34 AM, Steve Rowe <sarowe@gmail.com> wrote:
> 
>> Hi Joseph,
>> 
>> SOLR-6137 <https://issues.apache.org/jira/browse/SOLR-6137> fixed the NPE
>> you encountered with 4.9, as well a couple other managed schema concurrency
>> issues - it will be included in the next release of Solr: 4.10.
>> 
>> Steve
>> 
>> On Aug 15, 2014, at 11:00 AM, Joseph Obernberger <
>> joseph.obernberger@gmail.com> wrote:
>> 
>>> Hi - I've been using Solr Cloud in schema-less mode and am having some
>>> issues with 4.8.1 and 4.9.0 when adding lots of new fields.  In 4.8.1
>> I'll
>>> get continuous messages that say:
>>> 134567307 [qtp968427990-2492] INFO  org.apache.solr.schema.IndexSchema  â
>>> Failed to persist managed schema at /configs/M1/managed-schema - version
>>> mismatch
>>> 134567315 [qtp968427990-2492] INFO  org.apache.solr.schema.IndexSchema  â
>>> Failed to persist managed schema at /configs/M1/managed-schema - version
>>> mismatch
>>> 134567323 [qtp968427990-2492] INFO  org.apache.solr.schema.IndexSchema  â
>>> Failed to persist managed schema at /configs/M1/managed-schema - version
>>> mismatch
>>> 
>>> In 4.9.0, something similar happens, but it comes with a stack trace:
>>> 
>>> 710115 [qtp851206054-45] ERROR org.apache.solr.schema.IndexSchema  â
>> Error
>>> persisting managed schema at /configs/M1/managed-schema
>>> java.lang.NullPointerException
>>>       at
>>> 
>> org.apache.solr.schema.IndexSchema.getCopyFieldProperties(IndexSchema.java:1418)
>>>       at
>>> 
>> org.apache.solr.schema.IndexSchema.getNamedPropertyValues(IndexSchema.java:1375)
>>>       at
>> org.apache.solr.schema.IndexSchema.persist(IndexSchema.java:379)
>>>       at
>>> 
>> org.apache.solr.schema.ManagedIndexSchema.persistManagedSchemaToZooKeeper(ManagedIndexSchema.java:137)
>>>       at
>>> 
>> org.apache.solr.schema.ManagedIndexSchema.persistManagedSchema(ManagedIndexSchema.java:83)
>>>       at
>>> 
>> org.apache.solr.schema.ManagedIndexSchema.addFields(ManagedIndexSchema.java:244)
>>>       at
>>> 
>> org.apache.solr.schema.ManagedIndexSchema.addFields(ManagedIndexSchema.java:201)
>>>       at
>>> 
>> org.apache.solr.schema.ManagedIndexSchema.addFields(ManagedIndexSchema.java:50)
>>>       at
>>> 
>> org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory$AddSchemaFieldsUpdateProcessor.processAdd(AddSchemaFieldsUpdateProcessorFactory.java:318)
>>>       at
>>> 
>> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
>>>       at
>>> 
>> org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:117)
>>>       at
>>> 
>> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
>>>       at
>>> 
>> org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:117)
>>>       at
>>> 
>> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
>>>       at
>>> 
>> org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:117)
>>>       at
>>> 
>> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
>>>       at
>>> 
>> org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:117)
>>>       at
>>> 
>> org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
>>>       at
>>> 
>> org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:117)
>>>       at
>>> 
>> org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:96)
>>>       at
>>> 
>> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:166)
>>>       at
>>> 
>> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readIterator(JavaBinUpdateRequestCodec.java:136)
>>>       at
>>> org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:225)
>>>       at
>>> 
>> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamedList(JavaBinUpdateRequestCodec.java:121)
>>>       at
>>> org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:190)
>>>       at
>>> org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:116)
>>>       at
>>> 
>> org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:173)
>>>       at
>>> 
>> org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:106)
>>>       at
>>> org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:58)
>>>       at
>>> 
>> org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
>>>       at
>>> 
>> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
>>>       at
>>> 
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
>>>       at org.apache.solr.core.SolrCore.execute(SolrCore.java:1962)
>>>       at
>>> 
>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)
>>>       at
>>> 
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
>>>       at
>>> 
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
>>>       at
>>> 
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
>>>       at
>>> 
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
>>>       at
>>> 
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
>>>       at
>>> 
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
>>>       at
>>> 
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>>>       at
>>> 
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
>>>       at
>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
>>>       at
>>> 
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
>>>       at
>>> 
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
>>>       at
>>> 
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>>>       at
>>> 
>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
>>>       at
>>> 
>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
>>>       at
>>> 
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>>>       at org.eclipse.jetty.server.Server.handle(Server.java:368)
>>>       at
>>> 
>> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
>>>       at
>>> 
>> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
>>>       at
>>> 
>> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
>>>       at
>>> 
>> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
>>>       at
>> org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:953)
>>>      at
>>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
>>>       at
>>> 
>> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
>>>       at
>>> 
>> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
>>>       at
>>> 
>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>>>       at
>>> 
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>>>       at java.lang.Thread.run(Thread.java:745)
>>> 
>>> This happens when adding lots of new fields to a SolrInputDocument from
>>> multiple threads hitting the cluster (right now just 2 shards).
>>> Any ideas / work-arounds?  Thank you very much!
>>> 
>>> -Joe Obernberger
>> 
>> 


Mime
View raw message