manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lalit jangra <lalit.j.jan...@gmail.com>
Subject Re: Apache ManifoldCF job stuck up
Date Wed, 09 Jul 2014 14:56:10 GMT
Thanks Karl,

My idea was as its name suggests mcf-combined-war as i was trying to take
advantage of agent process inside war. My assumption is wrong here as i can
see here.

In this case, which war should i use - mcf-crawler-ui.war? If i use this or
any of other wars, then should i start agent process?

Please suggest.

Regards.


On Wed, Jul 9, 2014 at 3:48 PM, Karl Wright <daddywri@gmail.com> wrote:

> Hi Lalit,
>
> Probably it is not a database issue, but how locking is configured.
> Typically the combined war is run in a non-clustered environment with no
> external synchronization required (as it is in the example I pointed you
> at).  We don't currently have an example that uses combined war with
> external locking -- that would not make much sense.  Can you tell me why
> you are trying to do it that way?
>
> Karl
>
>
>
> On Wed, Jul 9, 2014 at 10:46 AM, lalit jangra <lalit.j.jangra@gmail.com>
> wrote:
>
>> Hi Karl,
>>
>> if i try ./start-combined.sh, the job runs and i could see it finishing
>> gracefully with updating index in solr.
>>
>> I assume there is some DB issue here? Any suggestions?
>>
>> Regards.
>>
>>
>> On Wed, Jul 9, 2014 at 3:17 PM, Karl Wright <daddywri@gmail.com> wrote:
>>
>>> Hi Lalit,
>>>
>>> Please do the following: In an unmodified dist/example directory, run
>>> the start-combined script.  Open the UI, create the connections, and try to
>>> run the job.  Does it work for you?
>>>
>>>
>>> Karl
>>>
>>>
>>>
>>> On Wed, Jul 9, 2014 at 10:09 AM, lalit jangra <lalit.j.jangra@gmail.com>
>>> wrote:
>>>
>>>> Thanks Karl,
>>>>
>>>> I have reconfigured everything and this time i am starting tomcat and
>>>> deploying mcf-combined-war. I created output connection, repository
>>>> connection & job. Still when i start my job, it does not go ahead and
stuck
>>>> at starting up status.
>>>>
>>>>
>>>> Regards.
>>>>
>>>>
>>>>
>>>> On Wed, Jul 9, 2014 at 12:49 PM, Karl Wright <daddywri@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Lalit,
>>>>>
>>>>> You do not use combined.war AND an agents process at the same time.
>>>>> The combined war INCLUDES the agents process.
>>>>>
>>>>> Karl
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jul 9, 2014 at 6:18 AM, lalit jangra <lalit.j.jangra@gmail.com
>>>>> > wrote:
>>>>>
>>>>>> Hi Karl,
>>>>>>
>>>>>> As per your suggestion, i built a new MCF 1.5.1 instance, con from
>>>>>> source , configured it and pointed to local postgresql DB in
>>>>>> properties.xml  under /multiprocess-file-example. Then i tool a brand
new
>>>>>> tomcat and configured for MCF in it in setenv.sh for properties.xml
using
>>>>>> "-Dorg.apache.manifoldcf.configfile" argument and deployed mcf-combined.war
>>>>>> using app manager.
>>>>>>
>>>>>> First i initialized DB using ./initialize.sh command & the moment
i
>>>>>> started agent using ./start-agents.sh command under
>>>>>> /multiprocess-file-example, i got the error, even when tomcat is
not
>>>>>> started at this time. I created a job which never runs & get
stuck.
>>>>>>
>>>>>>
>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exception tossed:
>>>>>>  Service 'A' of type
>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is already
>>>>>>   active
>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Service
>>>>>> 'A'  of type 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
is
>>>>>>  already active
>>>>>>
>>>>>> But if i try to run jetty bundle using start.jar under example, no
>>>>>> such error but it does not fulfil my requirements.
>>>>>>
>>>>>> Regards.
>>>>>>
>>>>>>
>>>>>> On Wed, Jul 9, 2014 at 10:03 AM, Karl Wright <daddywri@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> So, lalit, if you run the multiprocess example without any changes,
>>>>>>> do
>>>>>>> you see this?  I don't.
>>>>>>>
>>>>>>> Karl
>>>>>>>
>>>>>>> Sent from my Windows Phone
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: lalit jangra
>>>>>>> Sent: 7/9/2014 12:17 AM
>>>>>>> To: Karl Wright
>>>>>>> Cc: user@manifoldcf.apache.org
>>>>>>> Subject: Re: Apache ManifoldCF job stuck up
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks Karl,
>>>>>>>
>>>>>>>
>>>>>>> I am currently running only single agent process on single machine
>>>>>>> without clustering. I have two environments and i could see this
>>>>>>> issue
>>>>>>> coming up at both places.
>>>>>>>
>>>>>>> While trying to start agent, i could see below error. When i
am
>>>>>>> trying
>>>>>>> to start agent, it says below error and exits. But there is no
agent
>>>>>>> process already running.
>>>>>>>
>>>>>>>
>>>>>>> [root@server1 multiprocess-file-example]# ./start-agents.sh &
>>>>>>>
>>>>>>> [1] 5020
>>>>>>>
>>>>>>> [root@server1 multiprocess-file-example]# Running...
>>>>>>>
>>>>>>> Configuration file successfully read
>>>>>>>
>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Service
>>>>>>> 'A'
>>>>>>> of type 'AGENT' is already active
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.agents.AgentRun.doExecute(AgentRun.java:54)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.agents.BaseAgentsInitializationCommand.execute(BaseAgentsInitializationCommand.java:37)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.agents.AgentRun.main(AgentRun.java:93)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> [1]+  Exit 1                  ./start-agents.sh
>>>>>>>
>>>>>>>
>>>>>>> If i am able to successfully start agent using ./start-agents.sh,
>>>>>>> still it throws same error but i have no other process running.
>>>>>>>
>>>>>>> Regards.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jul 8, 2014 at 2:38 PM, Karl Wright <daddywri@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hi lalit,
>>>>>>>
>>>>>>> This occurs when you have more than one agents process with the
same
>>>>>>>  process id using the same shared file system directory / zookeeper
>>>>>>>  cluster.  There is no other way it can occur.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>  Karl
>>>>>>>
>>>>>>> Sent from my Windows Phone
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>>  From: lalit jangra
>>>>>>>  Sent: 7/8/2014 8:38 AM
>>>>>>>  To: user@manifoldcf.apache.org
>>>>>>>  Subject: Re: Apache ManifoldCF job stuck up
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks Karl,
>>>>>>>
>>>>>>>
>>>>>>> I have tried steps you suggested and it worked on one instance.
>>>>>>>
>>>>>>> But on another instance i am still not able to resolve this issue.
>>>>>>>  Along with steps you mentioned, i tried recreating DB instance
>>>>>>> again,
>>>>>>>  setting up new MCF instance , cleaning locks and then starting
>>>>>>> agents
>>>>>>>  first & then tomcat. But issue still persists.
>>>>>>>
>>>>>>> If i try to run ./start-agents.sh, i get this error for agent
A.
>>>>>>>
>>>>>>>
>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exception tossed:
>>>>>>>  Service 'A' of type
>>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is
already
>>>>>>>  active
>>>>>>>
>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Service
>>>>>>> 'A'
>>>>>>>  of type 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
is
>>>>>>>  already active
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> But if i try to run ./start-agents-2.sh, i see similar error
but
>>>>>>> agent B
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exception tossed:
>>>>>>>  Service 'B' of type
>>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is
already
>>>>>>>  active
>>>>>>>
>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Service
>>>>>>> 'B'
>>>>>>>  of type 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
is
>>>>>>>  already active
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Regards.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jul 7, 2014 at 4:55 PM, Karl Wright <daddywri@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Hi Lalit,
>>>>>>>
>>>>>>>
>>>>>>> If you are using file synchronization, you cannot expect MCF
to clean
>>>>>>>  up itself unless you shut it down cleanly.  You should be either
>>>>>>> using
>>>>>>>  ^C or plain kill, NEVER kill -9.  kill -9 will leave dangling
locks.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> To clean up dangling locks:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> - shut ALL manifoldcf processes and web apps down
>>>>>>>
>>>>>>>
>>>>>>> - run the lock-clean script
>>>>>>>
>>>>>>>
>>>>>>> - start up the processes again
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>  Zookeeper synchronization, by the way, does not have this kind
of
>>>>>>> problem.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>  Karl
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jul 7, 2014 at 11:49 AM, lalit jangra <
>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>>
>>>>>>> I configured MCF 1.5.1 to run with Postgresql DB and tomcat 7.
>>>>>>>  Initially i created all connections, created a an alfresco job
and
>>>>>>> it
>>>>>>>  all worked fine.
>>>>>>>
>>>>>>>
>>>>>>> Next for updates, i stopped, tomcat and agent process running.
Then i
>>>>>>>  updated CmisRepositoryConnector.java with my own code and run
"ant
>>>>>>>  build" at root of MCF. It updated all code and jars file.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Also properties.xml is reset under /dist/multiprocess-file/example
>>>>>>>  which i updated to connect to Postgresql DB and logging
>>>>>>>  configurations.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I started tomcat then agent process.  Finally i started job to
crawl
>>>>>>>  alfresco but it got stuck and moving on. I checked into
>>>>>>>  /dist/multiprocess-file/example/logs/manifoldcf.log file and
could
>>>>>>> see
>>>>>>>  below error.
>>>>>>>
>>>>>>>
>>>>>>> ERROR 2014-07-07 16:09:04,936 (Agents thread) - Exception tossed:
>>>>>>>  Service '' of type
>>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is
already
>>>>>>>  active
>>>>>>>
>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Service
''
>>>>>>>  of type 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
is
>>>>>>>  already active
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270)
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon$AgentsThread.run(AgentsDaemon.java:208)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I created another job but that got stuck too.Is DB got corrupt
due
>>>>>>> to rebuild?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Also is this the right way to build MCF (I hope its correct way).
Now
>>>>>>>  what should i do to fix this issue?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Please help.
>>>>>>>
>>>>>>> Regards,
>>>>>>>  Lalit Jangra.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>  Regards,
>>>>>>>  Lalit Jangra.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Regards,
>>>>>>> Lalit Jangra.
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Lalit Jangra.
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Lalit Jangra.
>>>>
>>>
>>>
>>
>>
>> --
>> Regards,
>> Lalit Jangra.
>>
>
>


-- 
Regards,
Lalit Jangra.

Mime
View raw message