manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Standen Guy <Guy.Stan...@uk.fujitsu.com>
Subject RE: Advice on which PostgreSQL to use with ManifoldCF 2.6
Date Fri, 03 Mar 2017 16:17:10 GMT
Hi Karl,
                Thanks for coming back so quickly. Unfortunately I wasn’t using a JCIFS
connection. One of the issues I was seeing was between a crawl of an intranet site (no explicit
throttling other than number of connections) and   scheduled crawl (every 5 mins) to a relational
DB via JDBC connector again no explicit throttling.  To simplify things both jobs  are using
a NULL output connection. Sometimes both the Web crawl and the JDBC connection can run together
but at other times 1 or both jobs will appear to lock up with just a few active documents
showing. When I get a lock up the mcf log contains errors like:

“DEBUG 2017-03-03 15:28:20,466 (Worker thread '5') - Exception Database exception: SQLException
doing query (40001): ERROR: could not serialize access due to read/write dependencies among
transactions”

See the attached log extract for a little more detail.

Any view why this might be happening?

Best Regards,

Guy






From: Karl Wright [mailto:daddywri@gmail.com]
Sent: 03 March 2017 11:27
To: user@manifoldcf.apache.org
Subject: Re: Advice on which PostgreSQL to use with ManifoldCF 2.6

Hi Guy,

A issue with concurrent jobs is known for jobs sharing the same JCIFS connection.  Is that
what you are using?  This has nothing to do with the version of Postgresql you are using;
it has to do with what "bins" documents are thought to come from.  There has been a recent
improvement for this issue, which will be released in April.  See https://issues.apache.org/jira/browse/CONNECTORS-1364.

The current version of MCF (2.6) supports Solr 6.x.

Thanks,
Karl


On Fri, Mar 3, 2017 at 5:27 AM, Standen Guy <Guy.Standen@uk.fujitsu.com<mailto:Guy.Standen@uk.fujitsu.com>>
wrote:
Hi Karl,
I am currently using MCF 2.0.1 with PostgreSQL 9.3.5 on Windows and have had some issues with
multiple jobs running concurrently.
I am considering upgrading to MCF 2.6 and to a newer version of PostgreSQL. Would you be able
to advise which version of PostgreSQL I should consider using with MCF 2.6 (e.g.  PostgreSQL
 9.3.16 or 9.6.2)

I am also considering upgrading from SOLR 4.10.3 to a newer version. The MCF compatibility
matrix mentions that compatibility has been tested to SOLR version 4.5.1. Do you have any
advice about compatibility with the newer versions of SOLR e.g. 6.4.1.

Best Regards

Guy


Unless otherwise stated, this email has been sent from Fujitsu Services Limited (registered
in England No 96056); Fujitsu EMEA PLC (registered in England No 2216100) both with registered
offices at: 22 Baker Street, London W1U 3BW; PFU (EMEA) Limited, (registered in England No
1578652) and Fujitsu Laboratories of Europe Limited (registered in England No. 4153469) both
with registered offices at: Hayes Park Central, Hayes End Road, Hayes, Middlesex, UB4 8FE.
This email is only for the use of its intended recipient. Its contents are subject to a duty
of confidence and may be privileged. Fujitsu does not guarantee that this email has not been
intercepted and amended or that it is virus-free.


Unless otherwise stated, this email has been sent from Fujitsu Services Limited (registered
in England No 96056); Fujitsu EMEA PLC (registered in England No 2216100) both with registered
offices at: 22 Baker Street, London W1U 3BW;  PFU (EMEA) Limited, (registered in England No
1578652) and Fujitsu Laboratories of Europe Limited (registered in England No. 4153469) both
with registered offices at: Hayes Park Central, Hayes End Road, Hayes, Middlesex, UB4 8FE.

This email is only for the use of its intended recipient. Its contents are subject to a duty
of confidence and may be privileged. Fujitsu does not guarantee that this email has not been
intercepted and amended or that it is virus-free.
Mime
View raw message