lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gregory Chanan (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SOLR-6249) Schema API changes return success before all cores are updated
Date Thu, 14 Aug 2014 22:31:18 GMT

    [ https://issues.apache.org/jira/browse/SOLR-6249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097812#comment-14097812
] 

Gregory Chanan edited comment on SOLR-6249 at 8/14/14 10:29 PM:
----------------------------------------------------------------

bq. Seems to me like less coordination here is best? In other words, instead of the core that
accepts the update request worrying about all the other cores that are going to see the update,
it just assumes it will be successful. If any of the remote cores fail in processing the update,
then they mark themselves as "down". This seems to solve the root of the problem raised by
this ticket, namely a replica using the wrong version of a managed schema. It's likely that
a replica that can't process a good update successfully will need some manual intervention
anyway (such as missing a JAR file or something).

Sure, less coordination is best.  I think there are multiple issues here, though:
1) how do we handle Schema API modifications failing on some replicas?
2) if i'm a client and I want to update the schema and send some updates that are dependent
on the updated schema, how do I do that?  Right now, it's a pain (and without a client-visible
version it may get more complicated if we add more schema APIs, e.g. removing fields).

I don't think the "wait for one replica" idea solves either of those issues.


was (Author: gchanan):
Seems to me like less coordination here is best? In other words, instead of the core that
accepts the update request worrying about all the other cores that are going to see the update,
it just assumes it will be successful. If any of the remote cores fail in processing the update,
then they mark themselves as "down". This seems to solve the root of the problem raised by
this ticket, namely a replica using the wrong version of a managed schema. It's likely that
a replica that can't process a good update successfully will need some manual intervention
anyway (such as missing a JAR file or something).

Sure, less coordination is best.  I think there are multiple issues here, though:
1) how do we handle Schema API modifications failing on some replicas?
2) if i'm a client and I want to update the schema and send some updates that are dependent
on the updated schema, how do I do that?  Right now, it's a pain (and without a client-visible
version it may get more complicated if we add more schema APIs, e.g. removing fields).

I don't think the "wait for one replica" idea solves either of those issues.

> Schema API changes return success before all cores are updated
> --------------------------------------------------------------
>
>                 Key: SOLR-6249
>                 URL: https://issues.apache.org/jira/browse/SOLR-6249
>             Project: Solr
>          Issue Type: Improvement
>          Components: Schema and Analysis, SolrCloud
>            Reporter: Gregory Chanan
>
> See SOLR-6137 for more details.
> The basic issue is that Schema API changes return success when the first core is updated,
but other cores asynchronously read the updated schema from ZooKeeper.
> So a client application could make a Schema API change and then index some documents
based on the new schema that may fail on other nodes.
> Possible fixes:
> 1) Make the Schema API calls synchronous
> 2) Give the client some ability to track the state of the schema.  They can already do
this to a certain extent by checking the Schema API on all the replicas and verifying that
the field has been added, though this is pretty cumbersome.  Maybe it makes more sense to
do this sort of thing on the collection level, i.e. Schema API changes return the zk version
to the client.  We add an API to return the current zk version.  On a replica, if the zk version
is >= the version the client has, the client knows that replica has at least seen the schema
change.  We could also provide an API to do the distribution and checking across the different
replicas of the collection so that clients don't need ot do that themselves.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message