lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hendrik Haddorp <>
Subject Re: ClusterStateMutator
Date Thu, 05 Jan 2017 12:30:56 GMT
The UI warning was quite easy to resolve. I'm currently testing Solr 
with HDFS but for some reason the core ended up on the local storage of 
the node. After a delete and restart the problem was gone.

On 05.01.2017 12:42, Hendrik Haddorp wrote:
> Right, I had to do that multiple times already when I restarted nodes 
> during collection creation. In such cases I was left with data in the 
> clusterstate.json, which at least on 6.2.1, blocked further collection 
> creations. Once manually deleted or set to {} collection creation 
> worked again.
> Setting legacyCloud=false looks good. I don't get anything in 
> clusterstate.json anymore and no old collections show up after a node 
> restarts. I could also confirm what Shalin said, that state format 2 
> is used by default. Only if I explicitly set state format to 1 I see 
> data in clusterstate.json during the collection creation. Just the 
> Solr admin UI is now showing "SolrCore Initialization Failures" 
> pointing to none existing replicas. I assume that happens when Solr 
> starts up and finds data for a core that does not exist in ZK anymore. 
> How would one clean up this issue? Beside that the some replicas can 
> still end up broken if the node restarts in the wrong time. I 
> currently have one replica marked as down and one as gone. So far I 
> was however always able to manually replace these replicas to resolve 
> this state. So in general this looks quite good now. Guess I will 
> still need to find a way to make sure that I don't restart a node 
> during collection creation :-(
> On 05.01.2017 02:33, Erick Erickson wrote:
>> Let us know how it goes. You'll probably want to remove the _contents_
>> of clusterstate.json and just leave it as a pair of brackets , i.e. {}
>> if for no other reason than it's confusing.
>> Times past the node needed to be there even if empty. Although I just
>> tried removing it completely on 6x and I was able to start Solr, part
>> of the startup process recreates it as an empty node, just a pair of
>> braces.
>> Best,
>> Erick
>> On Wed, Jan 4, 2017 at 1:22 PM, Hendrik Haddorp 
>> <> wrote:
>>> Hi Erik,
>>> I have actually also seen that behavior already. So will check what
>>> happens when I set that property.
>>> I still believe I'm getting the clusterstate.json set already before 
>>> the
>>> node comes up again. But I will try to verify that further tomorrow.
>>> thanks,
>>> Hendrik
>>> On 04/01/17 22:10, Erick Erickson wrote:
>>>> Hendrik:
>>>> Historically in 4.x, there was code that would reconstruct the
>>>> clusterstate.json code. So you would see "deleted" collections come
>>>> back. One scenario was:
>>>> - Have a Solr node offline that had a replica for a collection.
>>>> - Delete that collection
>>>> - Bring the node back
>>>> - It would register itself in clusterstate.json.
>>>> So my guess is that something like this is going on and you're getting
>>>> a clusterstate.json that's reconstructed (and possibly not complete).
>>>> You can avoid this by specifying legacyCloud=false clusterprop
>>>> Kind of a shot in the dark...
>>>> Erick
>>>> On Wed, Jan 4, 2017 at 11:12 AM, Hendrik Haddorp
>>>> <> wrote:
>>>>> You are right, the code looks like it. But why did I then see 
>>>>> collection
>>>>> data in the clusterstate.json file? If version 1 is not used I would
>>>>> assume that no data ends up in there. When explicitly setting the 
>>>>> state
>>>>> format 2 the system seemed to behave differently. And if the code 
>>>>> always
>>>>> uses version 2 shouldn't the default in that line be changed 
>>>>> accordingly?
>>>>> On 04/01/17 16:41, Shalin Shekhar Mangar wrote:
>>>>>> Actually the state format defaults to 2 since many releases (all
>>>>>> 6.x at least). This default is enforced in CollectionsHandler much
>>>>>> before the code in ClusterStateMutator is executed.
>>>>>> On Wed, Jan 4, 2017 at 6:16 PM, Hendrik Haddorp 
>>>>>> <> wrote:
>>>>>>> Hi,
>>>>>>> in
>>>>>>> solr-6.3.0/solr/core/src/java/org/apache/solr/cloud/overseer/

>>>>>>> there is the following code starting line 107:
>>>>>>> //TODO default to 2; but need to debug why 
>>>>>>> BasicDistributedZk2Test fails
>>>>>>> early on
>>>>>>>      String znode = message.getInt(DocCollection.STATE_FORMAT,

>>>>>>> 1) == 1 ? null
>>>>>>>          : ZkStateReader.getCollectionPath(cName);
>>>>>>> Any if that will be changed to default to version 2 anytime soon?
>>>>>>> thanks,
>>>>>>> Hendrik

View raw message