incubator-connectors-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Connectors Framework > How to Build and Deploy ManifoldCF
Date Thu, 04 Nov 2010 16:12:00 GMT
Space: Apache Connectors Framework (https://cwiki.apache.org/confluence/display/CONNECTORS)
Page: How to Build and Deploy ManifoldCF (https://cwiki.apache.org/confluence/display/CONNECTORS/How+to+Build+and+Deploy+ManifoldCF)
Comment: https://cwiki.apache.org/confluence/display/CONNECTORS/How+to+Build+and+Deploy+ManifoldCF?focusedCommentId=24185371#comment-24185371

Comment added by Farzad:
---------------------------------------------------------------------

The trunk of Solr has 14,752 files and 9,269 folders.  The job completd in 24 mintues and
11 seconds, or 1451 seconds.  I'm getting a rate of 16.6 items / sec. If I use only the files
the rate is 10.2 files / sec.  Did you use the total count or file count?

Is there a tool we can both use to compare the systems. Do you have ManifoldCF, database,
and appserver running off the same disk?  I first had the data on the same disk as Manifold,
then I moved it to a network drive, and the crawl time went up by 35 seconds.

My only goal at this point is to achieve your results on my system.

In reply to a comment by Karl Wright:
My test set is the lucene/solr trunk.  Check it out using svn and crawl that, including
the .svn directories, and see how fast it is for you.

Change your notification preferences: https://cwiki.apache.org/confluence/users/viewnotifications.action

Mime
View raw message