manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erlend Garåsen <e.f.gara...@usit.uio.no>
Subject Re: Unexpected job status encountered
Date Thu, 27 Dec 2018 10:42:01 GMT

It wasn't necessary to deal with tools like tcpdump etc. Adding the
following to the logging.xml did the trick:
<Logger name="org.apache.http.wire" level="debug" additivity="false">
  <AppenderRef ref="MyFile" />
</Logger>

So now I know what's going on. Bad credentials:

DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 <<
"HTTP/1.1 401 Unauthorized[\r][\n]"

Strange, because connection is working according to the Solr Output
Connector. I'll double-check whether the Solr server has another
password for index writing (path "/solr/uio/update/extract"). Or maybe
we have an encoding issue with the password since it's long and contains
special characters.

--8<--

DEBUG 2018-12-27T11:18:41,591 (Thread-1508) - http-outgoing-2 >>
"</div><!-- container --> [\n]"
DEBUG 2018-12-27T11:18:41,591 (Thread-1508) - http-outgoing-2 >>
"</body> [\n]"
DEBUG 2018-12-27T11:18:41,591 (Thread-1508) - http-outgoing-2 >>
"</html>[\n]"
DEBUG 2018-12-27T11:18:41,591 (Thread-1508) - http-outgoing-2 >> "[\n]"
DEBUG 2018-12-27T11:18:41,591 (Thread-1508) - http-outgoing-2 >> "[\r][\n]"
DEBUG 2018-12-27T11:18:41,592 (Thread-1508) - http-outgoing-2 >>
"2f[\r][\n]"
DEBUG 2018-12-27T11:18:41,592 (Thread-1508) - http-outgoing-2 >> "[\r][\n]"
DEBUG 2018-12-27T11:18:41,592 (Thread-1508) - http-outgoing-2 >>
"******************[\r][\n]"
DEBUG 2018-12-27T11:18:41,592 (Thread-1508) - http-outgoing-2 >> "[\r][\n]"
DEBUG 2018-12-27T11:18:41,592 (Thread-1508) - http-outgoing-2 >> "0[\r][\n]"
DEBUG 2018-12-27T11:18:41,592 (Thread-1508) - http-outgoing-2 >> "[\r][\n]"
DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 <<
"HTTP/1.1 401 Unauthorized[\r][\n]"
DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 << "Date:
Thu, 27 Dec 2018 10:18:41 GMT[\r][\n]"
DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 <<
"Server: Apache/2.4.6 (Red Hat Enterprise Linux)
OpenSSL/1.0.2k-fips[\r][\n]"
DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 <<
"WWW-Authenticate: Basic realm="Solr"[\r][\n]"
DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 <<
"Content-Length: 381[\r][\n]"
DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 <<
"Keep-Alive: timeout=10, max=100[\r][\n]"
DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 <<
"Connection: Keep-Alive[\r][\n]"
DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 <<
"Content-Type: text/html; charset=iso-8859-1[\r][\n]"
DEBUG 2018-12-27T11:18:41,593 (Thread-1508) - http-outgoing-2 << "[\r][\n]"
DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 <<
"<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">[\n]"
DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 <<
"<html><head>[\n]"
DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 <<
"<title>401 Unauthorized</title>[\n]"
DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 <<
"</head><body>[\n]"
DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 <<
"<h1>Unauthorized</h1>[\n]"
DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 <<
"<p>This server could not verify that you[\n]"
DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 << "are
authorized to access the document[\n]"
DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 <<
"requested.  Either you supplied the wrong[\n]"
DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 <<
"credentials (e.g., bad password), or your[\n]"
DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 <<
"browser doesn't understand how to supply[\n]"
DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 << "the
credentials required.</p>[\n]"
DEBUG 2018-12-27T11:18:41,594 (Thread-1508) - http-outgoing-2 <<
"</body></html>[\n]"
 WARN 2018-12-27T11:18:41,599 (Worker thread '48') - IO exception during
indexing https://www.journals.uio.no/index.php/Dialogia: null
org.apache.http.client.ClientProtocolException

Erlend

On 21/12/2018 16:39, Karl Wright wrote:
> I'll have a look as time permits, but it won't be for a couple of weeks.
> Usually it's best to debug with http wire debugging, not packet
> captures.  I'm not an HTTP expert so it may be better to bring this up
> to the HttpComponents/HttpClient list, not ManifoldCF.
> 
> Karl
> 
> 
> On Fri, Dec 21, 2018 at 8:54 AM Erlend Garåsen <e.f.garasen@usit.uio.no
> <mailto:e.f.garasen@usit.uio.no>> wrote:
> 
> 
>     I tried to configure preemptive authentication by modifying the
>     HttpPoster class, but I still get the same errors. Then I ran the
>     following command in order to analyze the traffic:
>     tcpdump -w /tmp/chatter.dmp -s 0 -i ens192 -X host
>     solr-test01.uio.no <http://solr-test01.uio.no>
> 
>     I'm not an expert in reading such outputs, but I can send you the dump
>     file, Karl. I don't want to let the file be available for others, even
>     though it's from our test environment.
> 
>     Erlend
> 
>     On 13/12/2018 15:09, Karl Wright wrote:
>     > Ok, thanks, I misunderstood where the SSL error was coming from.  The
>     > Solr connection is what is complaining.  Do you see "Connection
>     working"
>     > for your output connection?  Please forgive me if you already answered
>     > this; I didn't note it in your response.  If you see that, then the
>     > connector was able to talk to your Solr "ping" handler, which
>     would mean
>     > that the SSL configuration is right but there's something else
>     about the
>     > connection that we would have to figure out and deal with.
>     >
>     > Karl
>     >
>     >
>     > On Thu, Dec 13, 2018 at 8:58 AM Erlend Garåsen
>     <e.f.garasen@usit.uio.no <mailto:e.f.garasen@usit.uio.no>
>     > <mailto:e.f.garasen@usit.uio.no <mailto:e.f.garasen@usit.uio.no>>>
>     wrote:
>     >
>     >     On 13/12/2018 14:26, Karl Wright wrote:
>     >     > This is SSL.  Did you add the server's cert to the web
>     connector's
>     >     > keystore?  Or, if not, add a "trust all" rule?
>     >
>     >     Thanks for the reply, Karl.
>     >
>     >     Yes, the root certificate was added both in the repository
>     connection
>     >     and for the Solr connector.
>     >
>     >     I checked the "trust all" option and restarted the job. Same
>     problem.
>     >
>     >     BTW, I get a lot of 200s, so MCF is able to fetch the URLs.
>     >
>     >     I have attached a screenshot of what I can see in the simple
>     history
>     >     report.
>     >
>     >     Erlend
>     >
> 


Mime
View raw message