lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Create Collection Admin Request - unable to specify collection configName
Date Wed, 02 Dec 2015 04:49:56 GMT
Why are you adding the /solr to the -z parameter when you start Solr? You
never specify it for upconfig/downconfig and the like.

Of course I'm almost always in dev and have only a single ZK running.
And when I just
tried adding /solr the results were a completely useless admin screen
that hung there. So
I'm not entirely sure what's going on.

My suspicion is that your ZK configs are a mish-mash of starting with
the /solr and bare
ZK strings and thus is hard to untangle at this point. So here's what I'd do:

> Stop all your Zk instances and completely blow away the configs. These are usually in
/var/zookeeper

> make sure your Solr nodes are clean. By "clean" here I mean you don't have any
cores under server/solr. You probably don't, but I'd check.

> try your operations again without the /solr.

Of course you could just restart solr without the /solr on the -z
parameter for a quick
test, but even if that works I'd still blow away the ZK data just to
have a clean slate.

Best,
Erick

On Tue, Dec 1, 2015 at 6:29 PM, Kelly, Frank <frank.kelly@here.com> wrote:
> Thanks Erick,
>
> Yes I start Solr as follows
>
> $ ./bin/solr start -cloud -z
> 54.XX.XX.84:2181,54.XX.XX.124:2181,54.XX.XX.254:2181/solr -h 52.XX.XX.13
>
>
> And confirmed those are the ZooKeeper nodes in question.
>
> Good idea on the downconfig and I confirmed it is working e.g.
>
> $  ./server/scripts/cloud-scripts/zkcli.sh -cmd upconfig -confdir
> ./server/solr/configsets/scbe_public7_config -confname testForErick2 -z
> 54.XX.XX.124:2181
> $ ./server/scripts/cloud-scripts/zkcli.sh -cmd downconfig -confdir
> /tmp/erick2 -confname testForErick2 -z 54.XX.XX.84:2181
> $ ls -1 /tmp/erick2
> currency.xml
> lang
> protwords.txt
> _rest_managed.json
> schema.xml
> solrconfig.xml
> stopwords.txt
> synonyms.txt
>
> So this is telling me ZooKeeper is at least storing the configuration but
> something with Solr/Zk is awry.
>
> Cheers!
>
>
> -Frank
>
>
>
>
>
> On 12/1/15, 9:01 PM, "Erick Erickson" <erickerickson@gmail.com> wrote:
>
>>bq: I have tried upconfig to all three ZooKeeper nodes (1 leader, 2
>>followers)
>>and they seem to work but when I look at cloud/tree it doesn’t make a
>>difference.
>>
>>That doesn't answer how you start _Solr_. My question earlier was trying
>>to insure that your Solr instance points at the same zookeeper ensemble
>>as you hit when you upconfig.
>>
>>You should also be able to "downconfig" from the client and get a fresh
>>copy
>>from ZK to a new local directory to insure that it's really up in ZK.
>>
>>Best,
>>Erick
>>
>>On Tue, Dec 1, 2015 at 5:21 PM, Manikandan Sivanesan
>><msivanes@redhat.com> wrote:
>>> And one more thing I noticed you are specifying your ensemble as
>>> zk.zk.zk.zk:2181 . But it should be of the form
>>>*zk1:port,zk2:port,zk3:port
>>> *.
>>>
>>> On Tue, Dec 1, 2015 at 8:12 PM, Manikandan Sivanesan
>>><msivanes@redhat.com>
>>> wrote:
>>>
>>>>
>>>> Try with the conf directory in place with schema.xml & solrconfig.xml
>>>> inside conf.  You should see the files being uploaded
>>>>
>>>> ./server/scripts/cloud-scripts/zkcli.sh -cmd upconfig -confdir
>>>> *./server/solr/configsets/scbe_**public7_config/conf* -confname
>>>> scbe_public7 -z zk.zk.zk.zk:2181
>>>>
>>>> This is how I do
>>>> zkcli.sh  -zkhost $ZK_ENSEMBLE -cmd upconfig -confdir /tmp/access/conf
>>>> -confname  access
>>>>
>>>> You can verify if you have properly uploaded the config to either by
>>>> Upayvira's suggestion or using ./zkcli.sh Eg :
>>>> https://gist.github.com/manisnesan/52ffc84dd761365e0c22
>>>>
>>>>
>>>>
>>>> On Tue, Dec 1, 2015 at 5:22 PM, Kelly, Frank <frank.kelly@here.com>
>>>>wrote:
>>>>
>>>>> So I have an ensemble of three Zk nodes running
>>>>>
>>>>> I have tried upconfig to all three ZooKeeper nodes (1 leader, 2
>>>>>followers)
>>>>> and they seem to work but when I look at cloud/tree it doesn’t make
a
>>>>> difference.
>>>>> I tried switching browsers to avoid the potential browser caching
>>>>>issue
>>>>> and still no dice.
>>>>>
>>>>>
>>>>> The only thing I can think of is that “mycollection” which I *can*
>>>>>see is
>>>>> a Managed Schema - does having one ManagedSchema in ZooKeeper/Solr
>>>>> prevent Classic schema.xml based configurations from being configured
>>>>>-
>>>>> even if they are to be used by different indexes?
>>>>>
>>>>> I.e. Is the following possible
>>>>> config1 -> classic schema (schema.xml)
>>>>> config2 -> managed schema
>>>>>
>>>>> collection1 uses config1 (managed by schema.xml)
>>>>> collection2 uses config2 (managed by REST API)
>>>>>
>>>>> Best,
>>>>>
>>>>> -Frank
>>>>>
>>>>>
>>>>> Frank Kelly
>>>>> Principal Software Engineer
>>>>> Predictive Analytics Team (SCBE/HAC/CDA)
>>>>>
>>>>> Email: f <mailto:frank.kelly@here.com>r
>>>>> <mailto:frank.kelly@here.com>ank.kelly@here.com
>>>>> <mailto:frank.kelly@here.com>
>>>>> Website: http://www.here.com <http://www.here.com/>
>>>>>
>>>>> 5 Wayside Rd, Burlington, MA 01803, USA
>>>>> Here, a Nokia business
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 12/1/15, 5:10 PM, "Erick Erickson" <erickerickson@gmail.com>
wrote:
>>>>>
>>>>> >You shouldn't have to do the linkconfig to see the configs in the
ZK
>>>>> >tree, and it should appear virtually instantaneously. A couple of
>>>>> >possibilities:
>>>>> >
>>>>> >1> your browser is caching the view somehow (unlikely).
>>>>> >2> your Solr instance isn't pointing to the same ZK your upconfig
>>>>> >command is pointing to. This is easy to do if you start your Solr
>>>>>with
>>>>> >the bin scripts like 'bin/solr -c' start as that invokes the embedded
>>>>> >zookeeper, not the external ensemble, you have to use the -z option
>>>>> >for that (the -c option is unnecessary if you specify -z)
>>>>> >
>>>>> >And the first form of your CREATE command is what's required, i.e.
>>>>> >"collection.configName=scbe_public7"
>>>>> >not
>>>>> >"collection.configName=/configs/scbe_public7"
>>>>> >
>>>>> >Don't worry about the linkconfig, create or anything else until you
>>>>> >can see your configs in the tree view.
>>>>> >
>>>>> >Best,
>>>>> >Erick
>>>>> >
>>>>> >On Tue, Dec 1, 2015 at 1:53 PM, Kelly, Frank <frank.kelly@here.com>
>>>>> wrote:
>>>>> >> Thanks - the only thing under cloud/tree tab for /configs is
>>>>> >>mycollection
>>>>> >>
>>>>> >> But I am pretty sure my zookeeper config upload is working fine
>>>>> >>
>>>>> >> ./server/scripts/cloud-scripts/zkcli.sh -cmd upconfig -confdir
>>>>> >> ./server/solr/configsets/scbe_public7_config -confname
>>>>>scbe_public7 -z
>>>>> >> zk.zk.zk.zk:2181
>>>>> >> ./server/scripts/cloud-scripts/zkcli.sh -cmd linkconfig -collection
>>>>> >> scbe_public7 -confname scbe_public7 -z zk.zk.zk.zk:2181
>>>>> >>
>>>>> >> At least no errors were reported . . .
>>>>> >>
>>>>> >> I just re-ran the same commands and gave it a few minutes but
I
>>>>>still
>>>>> >> don¹t see the ³scbe_public7² config under /configs?
>>>>> >>
>>>>> >> -Frank
>>>>> >>
>>>>> >>
>>>>> >> On 12/1/15, 4:16 PM, "Upayavira" <uv@odoko.co.uk> wrote:
>>>>> >>
>>>>> >>>Check via the admin UI - cloud/tree tab. Check inside the
config
>>>>> >>>directory that you are attempting to use to see if there
is a
>>>>> >>>solrconfig.xml and a schema file. I've sometimes uploaded
it such
>>>>>that
>>>>> >>>there's a conf directory inside - i.e. one layer too deep.
>>>>> >>>
>>>>> >>>This is gonna be something trivial, I bet you!
>>>>> >>>
>>>>> >>>Upayavira
>>>>> >>>
>>>>> >>>On Tue, Dec 1, 2015, at 08:04 PM, Kelly, Frank wrote:
>>>>> >>>> Context: Solr 5.3.1 with ZooKeeper 3.4.6 (SolrCloud)
>>>>> >>>>
>>>>> >>>> Via the REST APU I am trying to create a collection
 and tie it
>>>>>to a
>>>>> >>>> configuration I have loaded into ZooKeeper
>>>>> >>>>
>>>>> >>>> Here are the configs loaded into ZooKeeper
>>>>> >>>>
>>>>> >>>> [zk: localhost:2181(CONNECTED) 5] ls /configs
>>>>> >>>> [scbe_public7, mycollection, scbe_public_conf]
>>>>> >>>>
>>>>> >>>> Here is my Curl command with scbe_public7 as the configName
>>>>> >>>>
>>>>> >>>> $ curl -X -v
>>>>> >>>>
>>>>> >>>>"
>>>>> http://XX.XX.XX.XX:8983/solr/admin/collections?action=CREATE&name=tmp2
>>>>> >>>>&n
>>>>>
>>>>>
>>>>>>>>>umShards=3&replicationFactor=2&maxShardsPerNode=1&collection.config
>>>>>>>>>Name
>>>>> >>>>=s
>>>>> >>>>cbe_public7"
>>>>> >>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>> >>>> <response>
>>>>> >>>> <lst name="responseHeader"><int name="status">400</int><int
>>>>> >>>> name="QTime">28</int></lst><str name="Operation
create caused
>>>>> >>>>
>>>>>
>>>>>
>>>>>>>>>exception:">org.apache.solr.common.SolrException:org.apache.solr.co
>>>>>>>>>mmon
>>>>> >>>>.S
>>>>> >>>>olrException:
>>>>> >>>> Can not find the specified config set: scbe_public7</str><lst
>>>>> >>>> name="exception"><str name="msg">Can not find
the specified
>>>>>config
>>>>> >>>>set:
>>>>> >>>> scbe_public7</str><int name="rspCode">400</int></lst><lst
>>>>> >>>> name="error"><str name="msg">Can not find the
specified config
>>>>>set:
>>>>> >>>> scbe_public7</str><int name="code">400</int></lst>
>>>>> >>>> </response>
>>>>> >>>>
>>>>> >>>> And if I try the following I get
>>>>> >>>>
>>>>> >>>> $ curl -X -v
>>>>> >>>>
>>>>> >>>>"
>>>>> http://XX.XX.XX.XX:8983/solr/admin/collections?action=CREATE&name=tmp2
>>>>> >>>>&n
>>>>>
>>>>>
>>>>>>>>>umShards=3&replicationFactor=2&maxShardsPerNode=1&collection.config
>>>>>>>>>Name
>>>>> >>>>=/
>>>>> >>>>configs/scbe_public7"
>>>>> >>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>> >>>> <response>
>>>>> >>>> <lst name="responseHeader"><int name="status">500</int><int
>>>>> >>>> name="QTime">29</int></lst><str name="Operation
create caused
>>>>> >>>>
>>>>>
>>>>>
>>>>>>>>>exception:">java.lang.IllegalArgumentException:java.lang.IllegalArg
>>>>>>>>>umen
>>>>> >>>>tE
>>>>> >>>>xception:
>>>>> >>>> Invalid path string "/configs//configs/scbe_public7"
caused by
>>>>>empty
>>>>> >>>>node
>>>>> >>>> name specified @9</str><lst name="exception"><str
>>>>>name="msg">Invalid
>>>>> >>>>path
>>>>> >>>> string "/configs//configs/scbe_public7" caused by empty
node name
>>>>> >>>> specified @9</str><int name="rspCode">-1</int></lst><lst
>>>>> >>>> name="error"><str name="msg">Invalid path string
>>>>> >>>> "/configs//configs/scbe_public7" caused by empty node
name
>>>>>specified
>>>>> >>>> @9</str><str name="trace">org.apache.solr.common.SolrException:
>>>>> >>>>Invalid
>>>>> >>>> path string "/configs//configs/scbe_public7" caused
by empty node
>>>>> name
>>>>> >>>> specified
>>>>> >>>>
>>>>> >>>> Interestingly it will let me create WITHOUT specifying
the config
>>>>> >>>>
>>>>> >>>>  curl -X -v
>>>>> >>>>
>>>>> >>>>"
>>>>> http://XX.XX.XX.XX:8983/solr/admin/collections?action=CREATE&name=tmp2
>>>>> >>>>&n
>>>>> >>>>umShards=3&replicationFactor=2&maxShardsPerNode=1"
>>>>> >>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>> >>>> <response>
>>>>> >>>> <lst name="responseHeader"><int name="status">0</int><int
>>>>> >>>> name="QTime">2699</int></lst><lst
name="success"><lst><lst
>>>>> >>>> name="responseHeader"><int name="status">0</int><int
>>>>> >>>> name="QTime">2042</int></lst><str
>>>>> >>>> name="core">tmp2_shard3_replica1</str></lst><lst><lst
>>>>> >>>> name="responseHeader"><int name="status">0</int><int
>>>>> >>>> name="QTime">2066</int></lst><str
>>>>> >>>> name="core">tmp2_shard1_replica1</str></lst><lst><lst
>>>>> >>>> name="responseHeader"><int name="status">0</int><int
>>>>> >>>> name="QTime">2088</int></lst><str
>>>>> >>>> name="core">tmp2_shard2_replica2</str></lst><lst><lst
>>>>> >>>> name="responseHeader"><int name="status">0</int><int
>>>>> >>>> name="QTime">2485</int></lst><str
>>>>> >>>> name="core">tmp2_shard3_replica2</str></lst><lst><lst
>>>>> >>>> name="responseHeader"><int name="status">0</int><int
>>>>> >>>> name="QTime">2531</int></lst><str
>>>>> >>>> name="core">tmp2_shard1_replica2</str></lst><lst><lst
>>>>> >>>> name="responseHeader"><int name="status">0</int><int
>>>>> >>>> name="QTime">2554</int></lst><str
>>>>> >>>> name="core">tmp2_shard2_replica1</str></lst></lst>
>>>>> >>>> </response>
>>>>> >>>>
>>>>> >>>> And it will be tied to (from CLUSTERSTATUS)
>>>>> >>>> "configName": "mycollection"
>>>>> >>>>
>>>>> >>>> This does not match the Documentation which states
>>>>> >>>>
>>>>> >>>>
>>>>>
>>>>>https://cwiki.apache.org/confluence/display/solr/Collections+API#Collec
>>>>> >>>>ti
>>>>> >>>>onsAPI-api1
>>>>> >>>>
>>>>> >>>> collection.configName "Defines the name of the configurations
>>>>>(which
>>>>> >>>>must
>>>>> >>>> already be stored in ZooKeeper) to use for this collection.
If
>>>>>not
>>>>> >>>> provided, Solr will default to the collection name as
the
>>>>> >>>>configuration
>>>>> >>>> name.²
>>>>> >>>>
>>>>> >>>> So I am confused on two things
>>>>> >>>>
>>>>> >>>>   1.  Why doesn¹t Solr see the ZooKeeper config I specified
(or
>>>>> >>>>   alternatively, is my syntax for the configName incorrect?)
>>>>> >>>>   2.  When the config is not specified why does it default
to
>>>>> >>>>   ³mycollection" and not to the collection name (and
thus cause
>>>>>an
>>>>> >>>>error
>>>>> >>>>   because ¹tmp2¹ is not a config )
>>>>> >>>>
>>>>> >>>> Thanks!
>>>>> >>>>
>>>>> >>>> -Frank
>>>>> >>>>
>>>>> >>>> Frank Kelly
>>>>> >>>> Principal Software Engineer
>>>>> >>>> Predictive Analytics Team (SCBE/HAC/CDA)
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> Email:
>>>>> >>>>
>>>>> >>>>f<mailto:frank.kelly@here.com>r<mailto:frank.kelly@here.com
>>>>> >ank.kelly@h
>>>>> >>>>er
>>>>> >>>>e.com<mailto:frank.kelly@here.com>
>>>>> >>>> Website: http://www.here.com<http://www.here.com/>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> 5 Wayside Rd, Burlington, MA 01803, USA
>>>>> >>>> Here, a Nokia business
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>
>>>>>
>>>>>
>>>>
>

Mime
View raw message