lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Gerlowski <gerlowsk...@gmail.com>
Subject Re: Intermittent error 401 with JSON Facet query to retrieve count all collections
Date Tue, 04 Jun 2019 12:15:29 GMT
Hi Edwin,

Thanks for the additional datapoint.  It seemed to work for me, but we
don't really understand the problem yet, so maybe it's not a solid
work around like I'd hoped.  I'm curious to hear whether it works for
Colvin.

To double check though: forwardCredentials is only supported in Solr >
8.0.  You're using an 8.x version, right?

Jason

On Tue, Jun 4, 2019 at 2:45 AM Zheng Lin Edwin Yeo <edwinyeozl@gmail.com> wrote:
>
> Hi Jason,
>
> Thanks for your reply.
>
> I have tried to add the "forwardCredentials": true in the security.json,
> but I still get the same error.
>
> Regards,
> Edwin
>
> On Mon, 3 Jun 2019 at 22:19, Colvin Cowie <colvin.cowie.dev@gmail.com>
> wrote:
>
> > Hi, thanks I'll give that a go when I get a chance.
> >
> > I was trying to reply to an older thread (
> >
> > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201904.mbox/%3CCAF2DzVXeVZqnixnkbzw0La1ui5N5-RG9PwfMBHG9vmkfBSMzJA%40mail.gmail.com%3E
> > ),
> > which I don't have in my mailbox, so obviously didn't reply to the right
> > address to get my response threaded so mine has appeared on its own. Oops.
> >
> > A JIRA issue was raised on that thread
> > https://issues.apache.org/jira/browse/SOLR-13421 but it's not had any
> > attention.
> >
> >
> > On Mon, 3 Jun 2019 at 14:46, Jason Gerlowski <gerlowskija@gmail.com>
> > wrote:
> >
> > > Hi Colvin,
> > >
> > > We're still taking a look at fixing the bug, but as a workaround in
> > > the meantime, you can look into adding a "forwardCredentials":true
> > > property under the "authentication" section of security.json.  That
> > > seems to fix the issue in my reproduction at least.
> > >
> > > e.g.
> > >
> > > {
> > >     "authentication": {
> > >         "blockUnknown": true,
> > >         "class": "solr.BasicAuthPlugin",
> > >         "credentials": {
> > >             "solradmin": "<encoded-pw>"
> > >         },
> > >         "forwardCredentials": true
> > >     },
> > >     ...
> > > }
> > >
> > > Jason
> > >
> > > On Mon, Jun 3, 2019 at 9:31 AM Jason Gerlowski <gerlowskija@gmail.com>
> > > wrote:
> > > >
> > > > One last note: as far as I can tell, nothing about this issue is
> > > > specific to JSON Faceting or the JSON request API.  It can be
> > > > triggered just as easily with "/select?q=*:*".
> > > >
> > > > The bug created for this is: SOLR-13510
> > > >
> > > > On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski <gerlowskija@gmail.com>
> > > wrote:
> > > > >
> > > > > I'm also able to reproduce this bug on master.  A few more notes
> > about
> > > > > the bad behavior:
> > > > >
> > > > > - the behavior occurs regardless of the specific permissions
> > > > > configured in security.json.  (i.e. whether the top permission is
> > > > > "all", or "security-edit", or there are no permissions at all.)
> > > > > - I tried looking for a pattern in which requests saw the 401s, but
> > > > > didn't have any luck.  The 401 occurs when talking to the whole
> > > > > collection or targeting individual cores directly.  It occurs when
> > > > > curl hits a host containing a replica for the collection in question,
> > > > > and when it doesnt. etc.  This distinguishes it from SOLR-13472,
> > which
> > > > > seems more specific to collection structure/layout.
> > > > >
> > > > > I'll create a bug for this in JIRA.
> > > > >
> > > > > On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie <
> > > colvin.cowie.dev@gmail.com> wrote:
> > > > > >
> > > > > > Hello. I encountered this issue too and wrote this up before
I
> > found
> > > this
> > > > > > thread, but I thought I might as well post it still, if it helps...
> > > > > >
> > > > > > Currently I'm trying to move our product on to Solr 8.1.1. We
are
> > > currently
> > > > > > using 6.6.6, so things have definitely moved on.
> > > > > >
> > > > > > We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to
lock
> > > down Solr
> > > > > > (and we also secure our zookeeper). Here's an example for solradmin
> > > as the
> > > > > > user and password
> > > > > >
> > > > > > {
> > > > > >     "authentication": {
> > > > > >         "blockUnknown": true,
> > > > > >         "class": "solr.BasicAuthPlugin",
> > > > > >         "credentials": {
> > > > > >             "solradmin":
> > > "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> > > > > > Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> > > > > >         }
> > > > > >     },
> > > > > >     "authorization": {
> > > > > >         "class": "solr.RuleBasedAuthorizationPlugin",
> > > > > >         "permissions": [
> > > > > >             {
> > > > > >                 "name": "all",
> > > > > >                 "role": "admin"
> > > > > >             }
> > > > > >         ],
> > > > > >         "user-role": {
> > > > > >             "solradmin": "admin"
> > > > > >         }
> > > > > >     }
> > > > > > }
> > > > > >
> > > > > >
> > > > > > On Solr 8.1.1, using our previously working security.json, running
> > > queries
> > > > > > (through the admin UI currently) I non-deterministically get
401
> > > responses
> > > > > > on queries when a collection has more than 1 shard. Increasing
the
> > > number
> > > > > > of shards in the collection makes the errors more likely.
> > > > > >
> > > > > > {
> > > > > >   "responseHeader":{
> > > > > >     "zkConnected":true,
> > > > > >     "status":401,
> > > > > >     "QTime":30,
> > > > > >     "params":{
> > > > > >       "q":"*:*",
> > > > > >       "_":"1559474550365"}},
> > > > > >   "error":{
> > > > > >     "metadata":[
> > > > > >
> > > > > >
> > >
> > "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
> > > > > >
> > > > > >
> > >
> > "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
> > > > > >     "msg":"Error from server at null: Expected mime type
> > > > > > application/octet-stream but got text/html. <html>\n<head>\n<meta
> > > > > > http-equiv=\"Content-Type\"
> > > > > > content=\"text/html;charset=utf-8\"/>\n<title>Error
401 require
> > > > > > authentication</title>\n</head>\n<body><h2>HTTP
ERROR
> > > 401</h2>\n<p>Problem
> > > > > > accessing /solr/gettingstarted_shard4_replica_n6/select.
> > > Reason:\n<pre>
> > > > > >  require authentication</pre></p>\n</body>\n</html>\n",
> > > > > >     "code":401}}
> > > > > >
> > > > > > The security stats indicate this is happening because the requests
> > > do not
> > > > > > have credentials with them, e.g.
> > > > > >
> > >
> > http://localhost:8983/solr/#/gettingstarted_shard4_replica_n6/plugins?type=security&entry=org.apache.solr.security.BasicAuthPlugin
> > > > > >
> > > > > >  org.apache.solr.security.BasicAuthPlugin
> > > > > >     class:
> > > > > >         org.apache.solr.security.BasicAuthPlugin
> > > > > >     description:
> > > > > >         Authentication Plugin
> > > org.apache.solr.security.BasicAuthPlugin
> > > > > >     stats
> > > > > >         SECURITY./authentication.authenticated:
> > > > > >             182
> > > > > >         SECURITY./authentication.errors.count:
> > > > > >             0
> > > > > >         SECURITY./authentication.failMissingCredentials:
> > > > > >             58
> > > > > >         SECURITY./authentication.failWrongCredentials:
> > > > > >             0
> > > > > >         SECURITY./authentication.passThrough:
> > > > > >             0
> > > > > >         SECURITY./authentication.requestTimes.meanRate:
> > > > > >             0.4183414110946125
> > > > > >         SECURITY./authentication.requests:
> > > > > >             240
> > > > > >         SECURITY./authentication.totalTime:
> > > > > >             117791100
> > > > > >
> > > > > > I assume that this is connected to the changes around
> > > > > > https://issues.apache.org/jira/browse/SOLR-7896 and
> > > > > > https://issues.apache.org/jira/browse/SOLR-13344 I've tested
with
> > > Solr
> > > > > > 7.6.0 and it appears to be unaffected
> > > > > >
> > > > > > Repro steps:
> > > > > >    # Extract solr 8.1.1.
> > > > > >    # bin\solr start -e cloud
> > > > > >         1 node / [default port] / [default collection name]
/ 4
> > > shards / 1
> > > > > > replica / [_default configuration]
> > > > > >    # server\scripts\cloud-scripts\zkcli -zkhost localhost:9983
-cmd
> > > putfile
> > > > > > /security.json <example-security.json file with content from
> > example
> > > above>
> > > > > >
> > > > > >    # Execute repeated GETS to
> > > > > > http://localhost:8983/solr/gettingstarted/select?q=*%3A* - a
lot
> > of
> > > them,
> > > > > > but not all, will fail with 401s
> > > > > >
> > > > > >
> > > > > > Also as a side note, because the authentication is now done
through
> > > the
> > > > > > form login rather than the browser basic auth, if you go directly
> > to
> > > a non
> > > > > > UI url (e.g. http://localhost:8983/solr/main_index/select?q=*%3A*)
> > > you have
> > > > > > to authenticate to it using the browser's basic auth prompt.
Which
> > is
> > > > > > slightly annoying since the query page in the Admin UI generates
> > > links to
> > > > > > it for the queries you run, and you've already authenticated
to get
> > > there.
> > > > > > But it's not a massive burden or anything... I guess I just
> > preferred
> > > > > > having the browser BA prompt.
> > > > > >
> > > > > > Thanks
> > >
> >

Mime
View raw message