Mailing-List: contact connectors-commits-help@incubator.apache.org;
 run by ezmlm
Precedence: bulk
Reply-To: connectors-dev@incubator.apache.org
Date: Mon, 15 Nov 2010 18:17:00 -0500 (EST)
From: confluence@apache.org
To: connectors-commits@incubator.apache.org
Message-ID: <229530.6564.1289863020028.JavaMail.confluence@thor>
Subject: [CONF] Apache Connectors Framework > How to Build and Deploy
 ManifoldCF
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Auto-Submitted: auto-generated

Space: Apache Connectors Framework (https://cwiki.apache.org/confluence/display/CONNECTORS)
Page: How to Build and Deploy ManifoldCF (https://cwiki.apache.org/confluence/display/CONNECTORS/How+to+Build+and+Deploy+ManifoldCF)
Comment: https://cwiki.apache.org/confluence/display/CONNECTORS/How+to+Build+and+Deploy+ManifoldCF?focusedCommentId=24186611#comment-24186611

Comment added by Farzad:
---------------------------------------------------------------------

Made the changes, system is still having problems, running but very slow.  I started the example with "java -Xms512m -Xmx102
4m -jar start.jar".  Got two java dumps at 40K and 80K, emailed them along with my config files to connectors-user@incubator.apache.org.  

In reply to a comment by Karl Wright:
It is clear then that your attempt to set up PostgreSQL with 400 database handles did not actually succeed, or my recommendation would not have helped.

The performance is still very poor compared with my very cheap system, but your disks now look reasonably quick.&nbsp; So let's try to figure out the problem.

(1) The default of 30 threads sounds low for your system.&nbsp; I'd up this to 100.

(2) You don't want the maximum connections to be a bottleneck, either on the repository connection side or on the output connection side.&nbsp; Set the max connections for both to 105.

(3) Configure your postgreSQL to have at least 200 database handles available.&nbsp; I know you tried to do this already, but for some reason you configuration did not work.

(4) Set your properties.xml maximum database connections parameter to be 105, so that's not a bottleneck either.


(5) You may want to give the JVM more memory than the default.&nbsp; Perhaps you are garbage collecting too much.&nbsp; If you are still using the quick-start, just add appropriate \-Xmx and \-Xms commands.&nbsp; I'd start with 1024MB.

If none of this helps, then we can figure out what the bottleneck is by getting a Java thread dump while the crawler is active.&nbsp; How you do this depends on what operating system you are using.&nbsp; But it should be possible from that thread dump to get an idea where all the threads are waiting.&nbsp; Post it to connectors-user@incubator.apache.org.

Thanks,
Karl

Change your notification preferences: https://cwiki.apache.org/confluence/users/viewnotifications.action