manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Schneider <scottsc...@gmail.com>
Subject Slow performance with a basic setup
Date Tue, 27 Mar 2012 22:24:10 GMT
Hi all,

I have a pretty simple ManifoldCF setup, but I'm getting very slow
performance.  Can someone help me understand and/or fix this?

My input is a web connector that goes to an Apache HTTP server running
on the local machine, serving static text files.  I have a null
authority service.  I output to Solr, also running locally.

The data I'm crawling is ~20 MB total in ~8,500 small files.  I start
the job one afternoon and the next morning, it was not finished!  It
had only processed ~2,500 documents.  Strangely, it listed ~10,000
total documents (and ~7,500 active).

My ultimate goal is to figure out how much space the Solr index takes
as I add more access tokens.  That's why I'm using the web connector
and null authority, rather than just using a file system connector.

Thanks,
Scott

Mime
View raw message