lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Dunning (Commented) (JIRA)" <>
Subject [jira] [Commented] (SOLR-2765) Shard/Node states
Date Sun, 09 Oct 2011 03:33:29 GMT


Ted Dunning commented on SOLR-2765:

Or, assuming state changes are infrequent, you just use optimistic locking and retries? 
Optimistic update (not locking) works great in ZK.  Just read, modify and then try to modify
the same version.  Repeat if the update fails.
Even if the whole state was an in XML file, I'm guessing it would be plenty faster though
... I would guess the real issue is the request per node. I'd guess the description of a large
cluster in one request would be much faster than reading a bit of data on 500 nodes.
On a local network, ZK requests are considerably less than millisecond.  For more distributed
networks, they are very near the round-trip time (20ms ... 1000 ms generally).  Reading one
file of 100K makes a difference of 100ms or less on most modern networks.  Reading many files
could be a real problem.  Getting a list of the znodes in a single directory should be almost
as fast as getting the contents of a single znode of the same size.

The only viable alternative really is to use the directory listing as the state.  That has
some bad consequences.  For instance, you may not want to use the host name as the only identifier
of a host.  You might want to include several hostnames.  You might want to include several
IP addresses.  You might want to have a hostname that includes slightly dangerous bytes (you
nuts if you do, though).  To do this with a directory listing is much trickier because you
usually need a dictionary off to the side to help you find all the alternative data while
still maintaining safe zknode names.  If you have the dictionary, that dictionary might as
well be the state file anyway and avoid the entire name/data problem.

> Shard/Node states
> -----------------
>                 Key: SOLR-2765
>                 URL:
>             Project: Solr
>          Issue Type: Sub-task
>          Components: SolrCloud, update
>            Reporter: Yonik Seeley
>             Fix For: 4.0
>         Attachments: incremental_update.patch, scheduled_executors.patch, shard-roles.patch
> Need state for shards that indicate they are recovering, active/enabled, or disabled.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message