incubator-connectors-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Connectors Framework > How to Build and Deploy ManifoldCF
Date Wed, 10 Nov 2010 23:57:00 GMT
Space: Apache Connectors Framework (https://cwiki.apache.org/confluence/display/CONNECTORS)
Page: How to Build and Deploy ManifoldCF (https://cwiki.apache.org/confluence/display/CONNECTORS/How+to+Build+and+Deploy+ManifoldCF)
Comment: https://cwiki.apache.org/confluence/display/CONNECTORS/How+to+Build+and+Deploy+ManifoldCF?focusedCommentId=24185965#comment-24185965

Comment added by Karl Wright:
---------------------------------------------------------------------

You do not need to increase the number of worker threads, unless you have a machine capable
of massive parallelism, in which case I could have sworn the instructions said it was OK to
increase the number of worker threads to around 100.  If you do that, you should have a postgresql
max connections parameter of 300 or 400.

The instructions for the postgresql parameters are specific to some extent to postgresql 8.x.
 I don't know what they should be on 9.x.  I can only tell you that your symptom is occurring
because ManifoldCF is trying to grab more connections that PostgreSQL is willing to give.
 Since ManifoldCF uses a connection pool, it can only grab one connection per thread.

The only way this would be violated is if a connector was written that uses connection-specific
database tables and doesn't properly return handles to the pool after grabbing them.  Is there
a possibility that this is occurring?

In reply to a comment by Farzad:
I didn't see any instructions talking about the need to increase the number of worker threads.
 Where is that listed?

Something doesn't add up, reading the conf, so each connection costs 400 bytes, each lock
cost 270 bytes.  The default max lock is 10 per connection.

10 x 400 x 270 = 1,080,000
400 x 400 = 160,000

Total shared memory needed = 1,240,000 bytes = which is 1.2 MB, we are setting the shared
memory to 1024MB, almost a 1000 more.  Why would there be a problem?

The other angle is I configured Postgre to have 400 connection but MCF is set to 30, not even
close.

Something is missing :)

Change your notification preferences: https://cwiki.apache.org/confluence/users/viewnotifications.action

Mime
View raw message