manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Detailed monitoring of jobs / job stuck
Date Mon, 17 Aug 2015 19:29:07 GMT
2.1 does do background reprioritization.  If you want to see that occurring
in the log, you would need to add the following in your properties.xml file:

<property name="org.apache.manifoldcf.scheduling" value="DEBUG"/>

Can I have more information?  Specifically, is this a multiprocess setup?
 and if so, is this zookeeper or file system synchronization?

Karl


On Mon, Aug 17, 2015 at 2:57 PM, Roman Šitina <roman@sitina.cz> wrote:

> Hello Karl,
>
> thanks for you quick reply!
>
> The version is 2.1. I tried to get detailed logging by setting
> log4j.rootLogger=INFO, MAIN in logging.ini but that did not help -
> only WARN level was still logging after restart.
>
> Roman
>
> On 17 August 2015 at 20:35, Karl Wright <daddywri@gmail.com> wrote:
> > Hi Roman,
> >
> > ManifoldCF needs to reprioritize documents whenever you pause or restart
> > jobs.  For jobs with large numbers of documents, the total amount of work
> > involved in this is significant.  But, depending on the precise
> ManifoldCF
> > version you are using, the reprioritization typically continues in
> > background while MCF runs your job.
> >
> > Can you tell me more about what version of MCF you are trying here?
> >
> > Karl
> >
> >
> > On Mon, Aug 17, 2015 at 2:13 PM, Roman Šitina <sitina@gmail.com> wrote:
> >>
> >> Hello,
> >>
> >> I have a ManifoldCF setup based on multiprocess-file-example which is
> >> backed by PostgreSQL.
> >>
> >> I have created a connection from Documentum to ElasticSearch with
> >> about 300 000 documents. I was able to crawl several thousand
> >> documents so the connection is working properly.
> >>
> >> What I'm not sure about is that when I pause or stop the job and then
> >> run it again it takes a while and it looks like ManifoldCF is doing
> >> nothing (30 minutes). After that time I usually try to restart all
> >> processes.
> >>
> >> I looked at all logs - manifoldcf.log, documentum-registry,
> >> documentum-server and DFC itself but I can't find any relevant
> >> information.
> >>
> >> Can you help me figuring out what is the best way to monitor progress
> >> of jobs that look to be not progressing?
> >>
> >> Thank you very much
> >> Roman
> >
> >
>

Mime
View raw message