lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kelly, Frank" <frank.ke...@here.com>
Subject Re: Create Collection Admin Request - unable to specify collection configName
Date Wed, 02 Dec 2015 02:29:43 GMT
Thanks Erick,

Yes I start Solr as follows

$ ./bin/solr start -cloud -z
54.XX.XX.84:2181,54.XX.XX.124:2181,54.XX.XX.254:2181/solr -h 52.XX.XX.13


And confirmed those are the ZooKeeper nodes in question.

Good idea on the downconfig and I confirmed it is working e.g.

$  ./server/scripts/cloud-scripts/zkcli.sh -cmd upconfig -confdir
./server/solr/configsets/scbe_public7_config -confname testForErick2 -z
54.XX.XX.124:2181
$ ./server/scripts/cloud-scripts/zkcli.sh -cmd downconfig -confdir
/tmp/erick2 -confname testForErick2 -z 54.XX.XX.84:2181
$ ls -1 /tmp/erick2
currency.xml
lang
protwords.txt
_rest_managed.json
schema.xml
solrconfig.xml
stopwords.txt
synonyms.txt

So this is telling me ZooKeeper is at least storing the configuration but
something with Solr/Zk is awry.

Cheers!


-Frank





On 12/1/15, 9:01 PM, "Erick Erickson" <erickerickson@gmail.com> wrote:

>bq: I have tried upconfig to all three ZooKeeper nodes (1 leader, 2
>followers)
>and they seem to work but when I look at cloud/tree it doesn’t make a
>difference.
>
>That doesn't answer how you start _Solr_. My question earlier was trying
>to insure that your Solr instance points at the same zookeeper ensemble
>as you hit when you upconfig.
>
>You should also be able to "downconfig" from the client and get a fresh
>copy
>from ZK to a new local directory to insure that it's really up in ZK.
>
>Best,
>Erick
>
>On Tue, Dec 1, 2015 at 5:21 PM, Manikandan Sivanesan
><msivanes@redhat.com> wrote:
>> And one more thing I noticed you are specifying your ensemble as
>> zk.zk.zk.zk:2181 . But it should be of the form
>>*zk1:port,zk2:port,zk3:port
>> *.
>>
>> On Tue, Dec 1, 2015 at 8:12 PM, Manikandan Sivanesan
>><msivanes@redhat.com>
>> wrote:
>>
>>>
>>> Try with the conf directory in place with schema.xml & solrconfig.xml
>>> inside conf.  You should see the files being uploaded
>>>
>>> ./server/scripts/cloud-scripts/zkcli.sh -cmd upconfig -confdir
>>> *./server/solr/configsets/scbe_**public7_config/conf* -confname
>>> scbe_public7 -z zk.zk.zk.zk:2181
>>>
>>> This is how I do
>>> zkcli.sh  -zkhost $ZK_ENSEMBLE -cmd upconfig -confdir /tmp/access/conf
>>> -confname  access
>>>
>>> You can verify if you have properly uploaded the config to either by
>>> Upayvira's suggestion or using ./zkcli.sh Eg :
>>> https://gist.github.com/manisnesan/52ffc84dd761365e0c22
>>>
>>>
>>>
>>> On Tue, Dec 1, 2015 at 5:22 PM, Kelly, Frank <frank.kelly@here.com>
>>>wrote:
>>>
>>>> So I have an ensemble of three Zk nodes running
>>>>
>>>> I have tried upconfig to all three ZooKeeper nodes (1 leader, 2
>>>>followers)
>>>> and they seem to work but when I look at cloud/tree it doesn’t make a
>>>> difference.
>>>> I tried switching browsers to avoid the potential browser caching
>>>>issue
>>>> and still no dice.
>>>>
>>>>
>>>> The only thing I can think of is that “mycollection” which I *can*
>>>>see is
>>>> a Managed Schema - does having one ManagedSchema in ZooKeeper/Solr
>>>> prevent Classic schema.xml based configurations from being configured
>>>>-
>>>> even if they are to be used by different indexes?
>>>>
>>>> I.e. Is the following possible
>>>> config1 -> classic schema (schema.xml)
>>>> config2 -> managed schema
>>>>
>>>> collection1 uses config1 (managed by schema.xml)
>>>> collection2 uses config2 (managed by REST API)
>>>>
>>>> Best,
>>>>
>>>> -Frank
>>>>
>>>>
>>>> Frank Kelly
>>>> Principal Software Engineer
>>>> Predictive Analytics Team (SCBE/HAC/CDA)
>>>>
>>>> Email: f <mailto:frank.kelly@here.com>r
>>>> <mailto:frank.kelly@here.com>ank.kelly@here.com
>>>> <mailto:frank.kelly@here.com>
>>>> Website: http://www.here.com <http://www.here.com/>
>>>>
>>>> 5 Wayside Rd, Burlington, MA 01803, USA
>>>> Here, a Nokia business
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 12/1/15, 5:10 PM, "Erick Erickson" <erickerickson@gmail.com> wrote:
>>>>
>>>> >You shouldn't have to do the linkconfig to see the configs in the ZK
>>>> >tree, and it should appear virtually instantaneously. A couple of
>>>> >possibilities:
>>>> >
>>>> >1> your browser is caching the view somehow (unlikely).
>>>> >2> your Solr instance isn't pointing to the same ZK your upconfig
>>>> >command is pointing to. This is easy to do if you start your Solr
>>>>with
>>>> >the bin scripts like 'bin/solr -c' start as that invokes the embedded
>>>> >zookeeper, not the external ensemble, you have to use the -z option
>>>> >for that (the -c option is unnecessary if you specify -z)
>>>> >
>>>> >And the first form of your CREATE command is what's required, i.e.
>>>> >"collection.configName=scbe_public7"
>>>> >not
>>>> >"collection.configName=/configs/scbe_public7"
>>>> >
>>>> >Don't worry about the linkconfig, create or anything else until you
>>>> >can see your configs in the tree view.
>>>> >
>>>> >Best,
>>>> >Erick
>>>> >
>>>> >On Tue, Dec 1, 2015 at 1:53 PM, Kelly, Frank <frank.kelly@here.com>
>>>> wrote:
>>>> >> Thanks - the only thing under cloud/tree tab for /configs is
>>>> >>mycollection
>>>> >>
>>>> >> But I am pretty sure my zookeeper config upload is working fine
>>>> >>
>>>> >> ./server/scripts/cloud-scripts/zkcli.sh -cmd upconfig -confdir
>>>> >> ./server/solr/configsets/scbe_public7_config -confname
>>>>scbe_public7 -z
>>>> >> zk.zk.zk.zk:2181
>>>> >> ./server/scripts/cloud-scripts/zkcli.sh -cmd linkconfig -collection
>>>> >> scbe_public7 -confname scbe_public7 -z zk.zk.zk.zk:2181
>>>> >>
>>>> >> At least no errors were reported . . .
>>>> >>
>>>> >> I just re-ran the same commands and gave it a few minutes but I
>>>>still
>>>> >> don¹t see the ³scbe_public7² config under /configs?
>>>> >>
>>>> >> -Frank
>>>> >>
>>>> >>
>>>> >> On 12/1/15, 4:16 PM, "Upayavira" <uv@odoko.co.uk> wrote:
>>>> >>
>>>> >>>Check via the admin UI - cloud/tree tab. Check inside the config
>>>> >>>directory that you are attempting to use to see if there is a
>>>> >>>solrconfig.xml and a schema file. I've sometimes uploaded it
such
>>>>that
>>>> >>>there's a conf directory inside - i.e. one layer too deep.
>>>> >>>
>>>> >>>This is gonna be something trivial, I bet you!
>>>> >>>
>>>> >>>Upayavira
>>>> >>>
>>>> >>>On Tue, Dec 1, 2015, at 08:04 PM, Kelly, Frank wrote:
>>>> >>>> Context: Solr 5.3.1 with ZooKeeper 3.4.6 (SolrCloud)
>>>> >>>>
>>>> >>>> Via the REST APU I am trying to create a collection  and
tie it
>>>>to a
>>>> >>>> configuration I have loaded into ZooKeeper
>>>> >>>>
>>>> >>>> Here are the configs loaded into ZooKeeper
>>>> >>>>
>>>> >>>> [zk: localhost:2181(CONNECTED) 5] ls /configs
>>>> >>>> [scbe_public7, mycollection, scbe_public_conf]
>>>> >>>>
>>>> >>>> Here is my Curl command with scbe_public7 as the configName
>>>> >>>>
>>>> >>>> $ curl -X -v
>>>> >>>>
>>>> >>>>"
>>>> http://XX.XX.XX.XX:8983/solr/admin/collections?action=CREATE&name=tmp2
>>>> >>>>&n
>>>>
>>>> 
>>>>>>>>umShards=3&replicationFactor=2&maxShardsPerNode=1&collection.config
>>>>>>>>Name
>>>> >>>>=s
>>>> >>>>cbe_public7"
>>>> >>>> <?xml version="1.0" encoding="UTF-8"?>
>>>> >>>> <response>
>>>> >>>> <lst name="responseHeader"><int name="status">400</int><int
>>>> >>>> name="QTime">28</int></lst><str name="Operation
create caused
>>>> >>>>
>>>>
>>>> 
>>>>>>>>exception:">org.apache.solr.common.SolrException:org.apache.solr.co
>>>>>>>>mmon
>>>> >>>>.S
>>>> >>>>olrException:
>>>> >>>> Can not find the specified config set: scbe_public7</str><lst
>>>> >>>> name="exception"><str name="msg">Can not find the
specified
>>>>config
>>>> >>>>set:
>>>> >>>> scbe_public7</str><int name="rspCode">400</int></lst><lst
>>>> >>>> name="error"><str name="msg">Can not find the specified
config
>>>>set:
>>>> >>>> scbe_public7</str><int name="code">400</int></lst>
>>>> >>>> </response>
>>>> >>>>
>>>> >>>> And if I try the following I get
>>>> >>>>
>>>> >>>> $ curl -X -v
>>>> >>>>
>>>> >>>>"
>>>> http://XX.XX.XX.XX:8983/solr/admin/collections?action=CREATE&name=tmp2
>>>> >>>>&n
>>>>
>>>> 
>>>>>>>>umShards=3&replicationFactor=2&maxShardsPerNode=1&collection.config
>>>>>>>>Name
>>>> >>>>=/
>>>> >>>>configs/scbe_public7"
>>>> >>>> <?xml version="1.0" encoding="UTF-8"?>
>>>> >>>> <response>
>>>> >>>> <lst name="responseHeader"><int name="status">500</int><int
>>>> >>>> name="QTime">29</int></lst><str name="Operation
create caused
>>>> >>>>
>>>>
>>>> 
>>>>>>>>exception:">java.lang.IllegalArgumentException:java.lang.IllegalArg
>>>>>>>>umen
>>>> >>>>tE
>>>> >>>>xception:
>>>> >>>> Invalid path string "/configs//configs/scbe_public7" caused
by
>>>>empty
>>>> >>>>node
>>>> >>>> name specified @9</str><lst name="exception"><str
>>>>name="msg">Invalid
>>>> >>>>path
>>>> >>>> string "/configs//configs/scbe_public7" caused by empty
node name
>>>> >>>> specified @9</str><int name="rspCode">-1</int></lst><lst
>>>> >>>> name="error"><str name="msg">Invalid path string
>>>> >>>> "/configs//configs/scbe_public7" caused by empty node name
>>>>specified
>>>> >>>> @9</str><str name="trace">org.apache.solr.common.SolrException:
>>>> >>>>Invalid
>>>> >>>> path string "/configs//configs/scbe_public7" caused by empty
node
>>>> name
>>>> >>>> specified
>>>> >>>>
>>>> >>>> Interestingly it will let me create WITHOUT specifying the
config
>>>> >>>>
>>>> >>>>  curl -X -v
>>>> >>>>
>>>> >>>>"
>>>> http://XX.XX.XX.XX:8983/solr/admin/collections?action=CREATE&name=tmp2
>>>> >>>>&n
>>>> >>>>umShards=3&replicationFactor=2&maxShardsPerNode=1"
>>>> >>>> <?xml version="1.0" encoding="UTF-8"?>
>>>> >>>> <response>
>>>> >>>> <lst name="responseHeader"><int name="status">0</int><int
>>>> >>>> name="QTime">2699</int></lst><lst name="success"><lst><lst
>>>> >>>> name="responseHeader"><int name="status">0</int><int
>>>> >>>> name="QTime">2042</int></lst><str
>>>> >>>> name="core">tmp2_shard3_replica1</str></lst><lst><lst
>>>> >>>> name="responseHeader"><int name="status">0</int><int
>>>> >>>> name="QTime">2066</int></lst><str
>>>> >>>> name="core">tmp2_shard1_replica1</str></lst><lst><lst
>>>> >>>> name="responseHeader"><int name="status">0</int><int
>>>> >>>> name="QTime">2088</int></lst><str
>>>> >>>> name="core">tmp2_shard2_replica2</str></lst><lst><lst
>>>> >>>> name="responseHeader"><int name="status">0</int><int
>>>> >>>> name="QTime">2485</int></lst><str
>>>> >>>> name="core">tmp2_shard3_replica2</str></lst><lst><lst
>>>> >>>> name="responseHeader"><int name="status">0</int><int
>>>> >>>> name="QTime">2531</int></lst><str
>>>> >>>> name="core">tmp2_shard1_replica2</str></lst><lst><lst
>>>> >>>> name="responseHeader"><int name="status">0</int><int
>>>> >>>> name="QTime">2554</int></lst><str
>>>> >>>> name="core">tmp2_shard2_replica1</str></lst></lst>
>>>> >>>> </response>
>>>> >>>>
>>>> >>>> And it will be tied to (from CLUSTERSTATUS)
>>>> >>>> "configName": "mycollection"
>>>> >>>>
>>>> >>>> This does not match the Documentation which states
>>>> >>>>
>>>> >>>>
>>>> 
>>>>https://cwiki.apache.org/confluence/display/solr/Collections+API#Collec
>>>> >>>>ti
>>>> >>>>onsAPI-api1
>>>> >>>>
>>>> >>>> collection.configName "Defines the name of the configurations
>>>>(which
>>>> >>>>must
>>>> >>>> already be stored in ZooKeeper) to use for this collection.
If
>>>>not
>>>> >>>> provided, Solr will default to the collection name as the
>>>> >>>>configuration
>>>> >>>> name.²
>>>> >>>>
>>>> >>>> So I am confused on two things
>>>> >>>>
>>>> >>>>   1.  Why doesn¹t Solr see the ZooKeeper config I specified
(or
>>>> >>>>   alternatively, is my syntax for the configName incorrect?)
>>>> >>>>   2.  When the config is not specified why does it default
to
>>>> >>>>   ³mycollection" and not to the collection name (and thus
cause
>>>>an
>>>> >>>>error
>>>> >>>>   because ¹tmp2¹ is not a config )
>>>> >>>>
>>>> >>>> Thanks!
>>>> >>>>
>>>> >>>> -Frank
>>>> >>>>
>>>> >>>> Frank Kelly
>>>> >>>> Principal Software Engineer
>>>> >>>> Predictive Analytics Team (SCBE/HAC/CDA)
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> Email:
>>>> >>>>
>>>> >>>>f<mailto:frank.kelly@here.com>r<mailto:frank.kelly@here.com
>>>> >ank.kelly@h
>>>> >>>>er
>>>> >>>>e.com<mailto:frank.kelly@here.com>
>>>> >>>> Website: http://www.here.com<http://www.here.com/>
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> 5 Wayside Rd, Burlington, MA 01803, USA
>>>> >>>> Here, a Nokia business
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>
>>>>
>>>>
>>>

Mime
View raw message