Return-Path: X-Original-To: apmail-manifoldcf-user-archive@www.apache.org Delivered-To: apmail-manifoldcf-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1349011EAC for ; Thu, 10 Jul 2014 10:00:13 +0000 (UTC) Received: (qmail 39308 invoked by uid 500); 10 Jul 2014 10:00:12 -0000 Delivered-To: apmail-manifoldcf-user-archive@manifoldcf.apache.org Received: (qmail 39254 invoked by uid 500); 10 Jul 2014 10:00:12 -0000 Mailing-List: contact user-help@manifoldcf.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@manifoldcf.apache.org Delivered-To: mailing list user@manifoldcf.apache.org Received: (qmail 39240 invoked by uid 99); 10 Jul 2014 10:00:12 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Jul 2014 10:00:12 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=AC_DIV_BONANZA,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of lalit.j.jangra@gmail.com designates 209.85.215.51 as permitted sender) Received: from [209.85.215.51] (HELO mail-la0-f51.google.com) (209.85.215.51) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Jul 2014 10:00:08 +0000 Received: by mail-la0-f51.google.com with SMTP id mc6so5956447lab.10 for ; Thu, 10 Jul 2014 02:59:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ez489HjNPlZ52tHmR48nozMLZCjIxvjuITkTkls7N7c=; b=dGaQflouP+YsxBd+Y5WpMUgZcTEhH7iSKYqyJN514161BURbl2u1GO1mpjpAyrIvvt L0peLCM/q6i1xH2a1ZB9ALopdkwOnb2TuQcguxCsnGMlJWWnf1iGYPOeugL7Th2Ytqij dHX5m3KP6S0mLRKZSr040J0Sl7k6ogQnnC8amDxBgG+Wr22r91lx/Rzi2payaM9WsR8i jv2lY25hJ1psgnGWMkommsSBKc87f6HIzIt8AW1ApEM4Wz0ryw4fd6+IUkYTL1/Vl7W7 g8XB6iGSeCRaC6D7Wbs1sZsCBN0Vu0R626C+i5k86RcTSrvj4WjW6cBmFroTFlxGCX9K fOoQ== MIME-Version: 1.0 X-Received: by 10.112.147.233 with SMTP id tn9mr3643071lbb.17.1404986386222; Thu, 10 Jul 2014 02:59:46 -0700 (PDT) Received: by 10.114.118.100 with HTTP; Thu, 10 Jul 2014 02:59:46 -0700 (PDT) In-Reply-To: References: <2810426633623221375@unknownmsgid> Date: Thu, 10 Jul 2014 10:59:46 +0100 Message-ID: Subject: Re: Apache ManifoldCF job stuck up From: lalit jangra To: Karl Wright Cc: "user@manifoldcf.apache.org" Content-Type: multipart/alternative; boundary=047d7b3a89787715b504fdd3e2b9 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b3a89787715b504fdd3e2b9 Content-Type: text/plain; charset=UTF-8 Thanks Karl, As you suggested, i followed below steps. 1. Stopped HSQL DB, agents & web applicaitons (by shutting down tomcat). 2. Updated properties.xml with below details for PostgreSQL & did leave all setting as it is. 3. Initialized DB with ./initialize.sh script, works fine and i can see successful registration messages. 4. Started agents using ./start-agents.sh 5. Started Tomcat, created connections & jobs.Started crawl job. 6. Job not taking off and still can see below error in logs. ERROR 2014-07-10 10:51:55,690 (Agents thread) - Exception tossed: Service 'A' of type 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is already active org.apache.manifoldcf.core.interfaces.ManifoldCFException: Service 'A' of type 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is already active Regards. On Thu, Jul 10, 2014 at 10:12 AM, Karl Wright wrote: > bq. So the only difference i could observe is DB which is HSQL vs > Postgres, rest everything is same. > > Right, exactly. Only this time we have some certainty that the > configuration is otherwise correct for synchronization and processes. > > The next step is the following: > > - START with the same multiprocess-file-example directory and processes > you verified yesterday > - Stop all MCF processes/web applications that are using that > multiprocess-file-example properties file: > - Either stop the Tomcat process, or better yet go to the "admin" tomcat > application and stop all three MCF web applications > - Go to the start-agents.sh window and ^C that process > - Go to the start-database.sh window and ^C that process > - Change the properties.xml file under multiprocess-file-example to point > to the PostgresDB instance you want it to use. This instance CANNOT be in > use by any other MCF cluster or process. > - Replace the following THREE properties in properties.xml with your > PostgreSQL equivalent, as follows: > > >>>>>> > value="org.apache.manifoldcf.core.database.DBInterfacePostgreSQL"/> > value=""/> > value=""/> > <<<<<< > > You may also choose to set these properties, to avoid collision with other > instances. You can use whatever value you like for these, within what > PostgreSQL allows: > > org.apache.manifoldcf.database.name > org.apache.manifoldcf.database.username > org.apache.manifoldcf.database.password > > - Run the initialize script: > - cd multiprocess-file-example ; initialize.sh . If this works, you > have the database configured properly. If it fails, look at the parameters > you provided and figure out what is wrong before proceeding > > - start the ManifoldCF processes and web applications: > - cd multiprocess-file-example ; start-agents.sh > - go to the tomcat instance, and start it, or start all the web > applications > > - create connections and jobs, and try a crawl. Let me know how it works. > > Karl > > > > On Thu, Jul 10, 2014 at 12:25 AM, lalit jangra > wrote: > >> Thanks a lot Karl, >> >> Yes, it works perfectly as you suggested in last mail. >> >> So the only difference i could observe is DB which is HSQL vs Postgres, >> rest everything is same. >> >> Please suggest. >> >> Regards. >> >> >> >> >> On Wed, Jul 9, 2014 at 11:35 PM, Karl Wright wrote: >> >>> Hi Lalit, >>> >>> >>> Sorry, I thought my instructions were clear. For this exercise, I want >>> you to use ONLY the example under dist/multiprocess-file-example. DO NOT >>> use the example under dist/example. >>> >>> Here are the instructions again, clarified and fleshed out: >>> >>> >>> - Start with a fresh build of MCF >>> - Go to the multiprocess-file example and start that in Jetty, with NO >>> changes: >>> - cd multiprocess-file-example >>> - start-database.sh >>> - initialize.sh >>> - start-agents.sh >>> - in a separate window: cd multiprocess-file-example ; start-webapps.sh >>> - create connections and job, see if it runs >>> - If that doesn't work, open a ticket. If that DOES work, proceed as >>> follows: >>> - Shut down the webapps process: >>> - go to the window where start-webapps.sh is running, and type ^C >>> - Set up a fresh tomcat instance >>> - Configure tomcat to point to the multiprocess-file example >>> properties.xml: >>> - Modify the tomcat startup scripts to include -D >>> org.apache.manifoldcf.configfile= >>> - Start the new instance of tomcat >>> - Deploy the wars to that tomcat (except combined war) >>> - Use the Tomcat admin tool to deploy mcf-crawler-ui.war, >>> mcf-authority-service.war, and mcf-api-service.war >>> - Go to the UI as served by Tomcat. You should see your connections, >>> job, etc just as before. Run the job. Does it work, yes or no? >>> >>> >>> Thanks, >>> Karl >>> >>> >>> >>> >>> >>> >>> >>> >>> On Wed, Jul 9, 2014 at 5:58 PM, lalit jangra >>> wrote: >>> >>>> Thanks Karl, >>>> >>>> Indeed i really appreciate your help and promptness. >>>> >>>> I tried the way you suggest and below are results. >>>> >>>> 1. Took a fresh MCF, started it using start.jar under dist/example >>>> using jetty. >>>> 2. Under multiprocess-file-example, initialized DB & started agent >>>> using .start-agents.sh. >>>> 3. Created all connection and jobs. Started job & it worked fine. >>>> 4. Next i shutdown jetty as started in step1. >>>> 5. Setup a fresh tomcat and pointed to >>>> multiprocess-file-example/properties.xml in its setenv.sh using MCF_HOME. >>>> 6. Started tomcat & deployed all but combined war. >>>> 7. Accessed mcf-crawler-ui war & accessed job. >>>> 8. On Job tab i get connection refused error. >>>> 9. If i start DB using ./start-databse.sh, i get "Error getting >>>> connection:invalid authorization exception-not found:manifoldcf" message. >>>> >>>> Really embarrassing for me as its not working. >>>> >>>> Regards. >>>> >>>> >>>> On Wed, Jul 9, 2014 at 6:52 PM, Karl Wright wrote: >>>> >>>>> Hi Lalit, >>>>> >>>>> Since I'm quite busy at work today, can you do the following: >>>>> >>>>> - Start with a fresh build of MCF >>>>> - Go to the multiprocess-file example and start that in Jetty, with NO >>>>> changes >>>>> - If that doesn't work, open a ticket. If that DOES work, proceed as >>>>> follows: >>>>> - Shut down the webapps process in the example >>>>> - Set up a fresh tomcat instance >>>>> - Configure tomcat to point to the multiprocess-file example >>>>> properties.xml. >>>>> - Deploy the wars to that tomcat (except combined war) >>>>> - Once again, try your job. Let me know the result. >>>>> >>>>> Thanks, >>>>> Karl >>>>> >>>>> >>>>> >>>>> On Wed, Jul 9, 2014 at 1:47 PM, Karl Wright >>>>> wrote: >>>>> >>>>>> Hi lalit, >>>>>> >>>>>> I don't see anything wrong with your environment. I suspect your >>>>>> configuration is what is wrong. >>>>>> >>>>>> Karl >>>>>> >>>>>> >>>>>> On Wed, Jul 9, 2014 at 1:46 PM, lalit jangra < >>>>>> lalit.j.jangra@gmail.com> wrote: >>>>>> >>>>>>> Thanks Karl, >>>>>>> >>>>>>> I agree with you. I am using Red Hat 6.4 with Postgres 9.2 & tomcat >>>>>>> 7. >>>>>>> >>>>>>> Can this be an issue ? >>>>>>> >>>>>>> Regards. >>>>>>> >>>>>>> >>>>>>> On Wed, Jul 9, 2014 at 6:43 PM, Karl Wright >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Lalit, >>>>>>>> >>>>>>>> I tried the zookeeper example here this morning and it worked >>>>>>>> perfectly. I am happy to try the file system example too, when I have a >>>>>>>> moment, but I very much doubt there is any problem with that code >>>>>>>> specifically. >>>>>>>> >>>>>>>> Remember that this is NOT a database issue. It is a >>>>>>>> synchronization issue ONLY. So it does not matter what database instance >>>>>>>> you point it to. >>>>>>>> >>>>>>>> Karl >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Jul 9, 2014 at 1:32 PM, lalit jangra < >>>>>>>> lalit.j.jangra@gmail.com> wrote: >>>>>>>> >>>>>>>>> Thanks Karl, >>>>>>>>> >>>>>>>>> I did it as you mentioned but still getting same error. This time >>>>>>>>> also i setup new DB and tried. Also i could see syncharea folder empty >>>>>>>>> after lock cleaning. >>>>>>>>> >>>>>>>>> Regards. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Jul 9, 2014 at 6:21 PM, Karl Wright >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Lalit, >>>>>>>>>> >>>>>>>>>> If you were using the multiprocess-file-example before, and you >>>>>>>>>> did not shut things down cleanly, you should do: ./lock-clean.sh first. >>>>>>>>>> >>>>>>>>>> Karl >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Jul 9, 2014 at 1:10 PM, lalit jangra < >>>>>>>>>> lalit.j.jangra@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Karl, >>>>>>>>>>> >>>>>>>>>>> I think its my bad luck today. :( >>>>>>>>>>> >>>>>>>>>>> I tried the way you just suggested but still getting same error >>>>>>>>>>> and job not able to run. What i did. >>>>>>>>>>> >>>>>>>>>>> 1. Created a brand new DB schema using ./executecommand.sh >>>>>>>>>>> org.apache.manifoldcf.core.DBCreate username password >>>>>>>>>>> 2. Initialized DB schema using ./initialize.sh >>>>>>>>>>> 3. Start agent process A using ./start-agents.sh >>>>>>>>>>> 4. Updated tomcat/bin/setenv.sh file to add >>>>>>>>>>> "-Dorg.apache.manifoldcf.configfile" parameter to point to properties.xml >>>>>>>>>>> under dist/multiprocess-file-example. >>>>>>>>>>> 5. Started Tomcat. >>>>>>>>>>> 6. Deployed all wars except combined war. >>>>>>>>>>> 7. Started job >>>>>>>>>>> >>>>>>>>>>> Job still not able to take off with same error in logs. >>>>>>>>>>> >>>>>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exception >>>>>>>>>>> tossed: Service 'A' of type >>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is already >>>>>>>>>>> active >>>>>>>>>>> >>>>>>>>>>> I think its not my day today. >>>>>>>>>>> >>>>>>>>>>> Regards. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, Jul 9, 2014 at 4:02 PM, Karl Wright >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Lalit, >>>>>>>>>>>> >>>>>>>>>>>> In theory the combined war will run with external locking, it's >>>>>>>>>>>> just that nobody ever thought that would be how people would use it, and >>>>>>>>>>>> therefore it hasn't been tested etc. >>>>>>>>>>>> >>>>>>>>>>>> I would suggest that for a development setup, you use the Jetty >>>>>>>>>>>> simple example. For a production setup, you should adopt the standard >>>>>>>>>>>> distributed arrangement as follows: >>>>>>>>>>>> >>>>>>>>>>>> - a deployed mcf-crawler-ui war >>>>>>>>>>>> - a deployed mcf-authority-service war >>>>>>>>>>>> - an (optional) deployed mcf-api-service war >>>>>>>>>>>> - one or more agents processes >>>>>>>>>>>> >>>>>>>>>>>> This is how it is done in both kinds of multiprocess examples, >>>>>>>>>>>> and is described thoroughly in the how-to-build-and-deploy document online. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Karl >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Jul 9, 2014 at 10:56 AM, lalit jangra < >>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Thanks Karl, >>>>>>>>>>>>> >>>>>>>>>>>>> My idea was as its name suggests mcf-combined-war as i was >>>>>>>>>>>>> trying to take advantage of agent process inside war. My assumption is >>>>>>>>>>>>> wrong here as i can see here. >>>>>>>>>>>>> >>>>>>>>>>>>> In this case, which war should i use - mcf-crawler-ui.war? If >>>>>>>>>>>>> i use this or any of other wars, then should i start agent process? >>>>>>>>>>>>> >>>>>>>>>>>>> Please suggest. >>>>>>>>>>>>> >>>>>>>>>>>>> Regards. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Jul 9, 2014 at 3:48 PM, Karl Wright < >>>>>>>>>>>>> daddywri@gmail.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Lalit, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Probably it is not a database issue, but how locking is >>>>>>>>>>>>>> configured. Typically the combined war is run in a non-clustered >>>>>>>>>>>>>> environment with no external synchronization required (as it is in the >>>>>>>>>>>>>> example I pointed you at). We don't currently have an example that uses >>>>>>>>>>>>>> combined war with external locking -- that would not make much sense. Can >>>>>>>>>>>>>> you tell me why you are trying to do it that way? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Karl >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 10:46 AM, lalit jangra < >>>>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Karl, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> if i try ./start-combined.sh, the job runs and i could see >>>>>>>>>>>>>>> it finishing gracefully with updating index in solr. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I assume there is some DB issue here? Any suggestions? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Regards. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 3:17 PM, Karl Wright < >>>>>>>>>>>>>>> daddywri@gmail.com> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi Lalit, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Please do the following: In an unmodified dist/example >>>>>>>>>>>>>>>> directory, run the start-combined script. Open the UI, create the >>>>>>>>>>>>>>>> connections, and try to run the job. Does it work for you? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Karl >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 10:09 AM, lalit jangra < >>>>>>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks Karl, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I have reconfigured everything and this time i am starting >>>>>>>>>>>>>>>>> tomcat and deploying mcf-combined-war. I created output connection, >>>>>>>>>>>>>>>>> repository connection & job. Still when i start my job, it does not go >>>>>>>>>>>>>>>>> ahead and stuck at starting up status. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Regards. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 12:49 PM, Karl Wright < >>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi Lalit, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> You do not use combined.war AND an agents process at the >>>>>>>>>>>>>>>>>> same time. The combined war INCLUDES the agents process. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Karl >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 6:18 AM, lalit jangra < >>>>>>>>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Hi Karl, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> As per your suggestion, i built a new MCF 1.5.1 >>>>>>>>>>>>>>>>>>> instance, con from source , configured it and pointed to local postgresql >>>>>>>>>>>>>>>>>>> DB in properties.xml under /multiprocess-file-example. Then i tool a brand >>>>>>>>>>>>>>>>>>> new tomcat and configured for MCF in it in setenv.sh for properties.xml >>>>>>>>>>>>>>>>>>> using "-Dorg.apache.manifoldcf.configfile" argument and deployed >>>>>>>>>>>>>>>>>>> mcf-combined.war using app manager. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> First i initialized DB using ./initialize.sh command & >>>>>>>>>>>>>>>>>>> the moment i started agent using ./start-agents.sh command under >>>>>>>>>>>>>>>>>>> /multiprocess-file-example, i got the error, even when tomcat is not >>>>>>>>>>>>>>>>>>> started at this time. I created a job which never runs & get stuck. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) - >>>>>>>>>>>>>>>>>>> Exception tossed: Service 'A' of type >>>>>>>>>>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is already >>>>>>>>>>>>>>>>>>> active >>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: >>>>>>>>>>>>>>>>>>> Service 'A' of type >>>>>>>>>>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is already active >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> But if i try to run jetty bundle using start.jar under >>>>>>>>>>>>>>>>>>> example, no such error but it does not fulfil my requirements. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Regards. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 10:03 AM, Karl Wright < >>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> So, lalit, if you run the multiprocess example without >>>>>>>>>>>>>>>>>>>> any changes, do >>>>>>>>>>>>>>>>>>>> you see this? I don't. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Karl >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Sent from my Windows Phone >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>>>>>>>>>> From: lalit jangra >>>>>>>>>>>>>>>>>>>> Sent: 7/9/2014 12:17 AM >>>>>>>>>>>>>>>>>>>> To: Karl Wright >>>>>>>>>>>>>>>>>>>> Cc: user@manifoldcf.apache.org >>>>>>>>>>>>>>>>>>>> Subject: Re: Apache ManifoldCF job stuck up >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks Karl, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I am currently running only single agent process on >>>>>>>>>>>>>>>>>>>> single machine >>>>>>>>>>>>>>>>>>>> without clustering. I have two environments and i could >>>>>>>>>>>>>>>>>>>> see this issue >>>>>>>>>>>>>>>>>>>> coming up at both places. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> While trying to start agent, i could see below error. >>>>>>>>>>>>>>>>>>>> When i am trying >>>>>>>>>>>>>>>>>>>> to start agent, it says below error and exits. But >>>>>>>>>>>>>>>>>>>> there is no agent >>>>>>>>>>>>>>>>>>>> process already running. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> [root@server1 multiprocess-file-example]# >>>>>>>>>>>>>>>>>>>> ./start-agents.sh & >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> [1] 5020 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> [root@server1 multiprocess-file-example]# Running... >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Configuration file successfully read >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: >>>>>>>>>>>>>>>>>>>> Service 'A' >>>>>>>>>>>>>>>>>>>> of type 'AGENT' is already active >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.AgentRun.doExecute(AgentRun.java:54) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.BaseAgentsInitializationCommand.execute(BaseAgentsInitializationCommand.java:37) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.AgentRun.main(AgentRun.java:93) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> [1]+ Exit 1 ./start-agents.sh >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> If i am able to successfully start agent using >>>>>>>>>>>>>>>>>>>> ./start-agents.sh, >>>>>>>>>>>>>>>>>>>> still it throws same error but i have no other process >>>>>>>>>>>>>>>>>>>> running. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Regards. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Tue, Jul 8, 2014 at 2:38 PM, Karl Wright < >>>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Hi lalit, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> This occurs when you have more than one agents process >>>>>>>>>>>>>>>>>>>> with the same >>>>>>>>>>>>>>>>>>>> process id using the same shared file system directory >>>>>>>>>>>>>>>>>>>> / zookeeper >>>>>>>>>>>>>>>>>>>> cluster. There is no other way it can occur. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> Karl >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Sent from my Windows Phone >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>>>>>>>>>> From: lalit jangra >>>>>>>>>>>>>>>>>>>> Sent: 7/8/2014 8:38 AM >>>>>>>>>>>>>>>>>>>> To: user@manifoldcf.apache.org >>>>>>>>>>>>>>>>>>>> Subject: Re: Apache ManifoldCF job stuck up >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks Karl, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I have tried steps you suggested and it worked on one >>>>>>>>>>>>>>>>>>>> instance. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> But on another instance i am still not able to resolve >>>>>>>>>>>>>>>>>>>> this issue. >>>>>>>>>>>>>>>>>>>> Along with steps you mentioned, i tried recreating DB >>>>>>>>>>>>>>>>>>>> instance again, >>>>>>>>>>>>>>>>>>>> setting up new MCF instance , cleaning locks and then >>>>>>>>>>>>>>>>>>>> starting agents >>>>>>>>>>>>>>>>>>>> first & then tomcat. But issue still persists. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> If i try to run ./start-agents.sh, i get this error for >>>>>>>>>>>>>>>>>>>> agent A. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) - >>>>>>>>>>>>>>>>>>>> Exception tossed: >>>>>>>>>>>>>>>>>>>> Service 'A' of type >>>>>>>>>>>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' >>>>>>>>>>>>>>>>>>>> is already >>>>>>>>>>>>>>>>>>>> active >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: >>>>>>>>>>>>>>>>>>>> Service 'A' >>>>>>>>>>>>>>>>>>>> of type >>>>>>>>>>>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is >>>>>>>>>>>>>>>>>>>> already active >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> But if i try to run ./start-agents-2.sh, i see similar >>>>>>>>>>>>>>>>>>>> error but agent B >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) - >>>>>>>>>>>>>>>>>>>> Exception tossed: >>>>>>>>>>>>>>>>>>>> Service 'B' of type >>>>>>>>>>>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' >>>>>>>>>>>>>>>>>>>> is already >>>>>>>>>>>>>>>>>>>> active >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: >>>>>>>>>>>>>>>>>>>> Service 'B' >>>>>>>>>>>>>>>>>>>> of type >>>>>>>>>>>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is >>>>>>>>>>>>>>>>>>>> already active >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Regards. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Mon, Jul 7, 2014 at 4:55 PM, Karl Wright < >>>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Hi Lalit, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> If you are using file synchronization, you cannot >>>>>>>>>>>>>>>>>>>> expect MCF to clean >>>>>>>>>>>>>>>>>>>> up itself unless you shut it down cleanly. You should >>>>>>>>>>>>>>>>>>>> be either using >>>>>>>>>>>>>>>>>>>> ^C or plain kill, NEVER kill -9. kill -9 will leave >>>>>>>>>>>>>>>>>>>> dangling locks. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> To clean up dangling locks: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> - shut ALL manifoldcf processes and web apps down >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> - run the lock-clean script >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> - start up the processes again >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Zookeeper synchronization, by the way, does not have >>>>>>>>>>>>>>>>>>>> this kind of problem. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> Karl >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Mon, Jul 7, 2014 at 11:49 AM, lalit jangra < >>>>>>>>>>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I configured MCF 1.5.1 to run with Postgresql DB and >>>>>>>>>>>>>>>>>>>> tomcat 7. >>>>>>>>>>>>>>>>>>>> Initially i created all connections, created a an >>>>>>>>>>>>>>>>>>>> alfresco job and it >>>>>>>>>>>>>>>>>>>> all worked fine. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Next for updates, i stopped, tomcat and agent process >>>>>>>>>>>>>>>>>>>> running. Then i >>>>>>>>>>>>>>>>>>>> updated CmisRepositoryConnector.java with my own code >>>>>>>>>>>>>>>>>>>> and run "ant >>>>>>>>>>>>>>>>>>>> build" at root of MCF. It updated all code and jars >>>>>>>>>>>>>>>>>>>> file. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Also properties.xml is reset under >>>>>>>>>>>>>>>>>>>> /dist/multiprocess-file/example >>>>>>>>>>>>>>>>>>>> which i updated to connect to Postgresql DB and logging >>>>>>>>>>>>>>>>>>>> configurations. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I started tomcat then agent process. Finally i started >>>>>>>>>>>>>>>>>>>> job to crawl >>>>>>>>>>>>>>>>>>>> alfresco but it got stuck and moving on. I checked into >>>>>>>>>>>>>>>>>>>> /dist/multiprocess-file/example/logs/manifoldcf.log >>>>>>>>>>>>>>>>>>>> file and could see >>>>>>>>>>>>>>>>>>>> below error. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ERROR 2014-07-07 16:09:04,936 (Agents thread) - >>>>>>>>>>>>>>>>>>>> Exception tossed: >>>>>>>>>>>>>>>>>>>> Service '' of type >>>>>>>>>>>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' >>>>>>>>>>>>>>>>>>>> is already >>>>>>>>>>>>>>>>>>>> active >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: >>>>>>>>>>>>>>>>>>>> Service '' >>>>>>>>>>>>>>>>>>>> of type >>>>>>>>>>>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is >>>>>>>>>>>>>>>>>>>> already active >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon$AgentsThread.run(AgentsDaemon.java:208) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I created another job but that got stuck too.Is DB got >>>>>>>>>>>>>>>>>>>> corrupt due to rebuild? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Also is this the right way to build MCF (I hope its >>>>>>>>>>>>>>>>>>>> correct way). Now >>>>>>>>>>>>>>>>>>>> what should i do to fix this issue? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Please help. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>>>> Lalit Jangra. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>>>> Lalit Jangra. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>>>> Lalit Jangra. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>>> Lalit Jangra. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>> Lalit Jangra. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>> Lalit Jangra. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Lalit Jangra. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Regards, >>>>>>>>>>> Lalit Jangra. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Regards, >>>>>>>>> Lalit Jangra. >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> Lalit Jangra. >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Lalit Jangra. >>>> >>> >>> >> >> >> -- >> Regards, >> Lalit Jangra. >> > > -- Regards, Lalit Jangra. --047d7b3a89787715b504fdd3e2b9 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks Karl,

As you s= uggested, i followed below steps.

1. Stopped HSQL DB, agents &= amp; web applicaitons (by shutting down tomcat).
2. Updated proper= ties.xml with below details for PostgreSQL & did leave all setting as i= t is.
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=20

=C2=A0 <property name=3D"org.apache.manifoldcf.databaseimplementationclass= " value=3D"org.apache.manifoldcf.core.database.DBInterfacePostgreSQL&quo= t;/>

=C2=A0 <property name=3D"org.apache.manifoldcf.dbsuperusername" value=3D"postgres"/>

=C2=A0 <property name=3D"org.apache.manifoldcf.dbsuperuserpassword" v= alue=3D"admin"/>

=C2=A0 <property name=3D"org.apache.manifoldcf.database.name" value=3D"mcfdkw" />

=C2=A0

=C2=A0 <property name=3D"org.apache.manifoldcf.hsqldbdatabaseprotocol"= ; value=3D"hsql"/>

=C2=A0 <property name=3D"org.apache.manifoldcf.hsqldbdatabaseserver" value=3D"localhost"/>

=C2=A0 <property name=3D"org.apache.manifoldcf.hsqldbdatabaseinstance"= ; value=3D"xdb"/>

=C2=A0 <property name=3D"org.apache.manifoldcf.database.maxhandles" value=3D"500"/>

=C2=A0 <property name=3D"org.apache.manifoldcf.crawler.threads" value=3D"500"/>

=C2=A0

3. Initialized DB with ./initialize.sh script, works fine and i can see suc= cessful registration messages.
4. Started agents using ./start-age= nts.sh
5. Started Tomcat, created connections & jobs.Started c= rawl job.
6.=C2=A0 Job not taking off and still can see below error in logs.
=C2= =A0=C2=A0=C2=A0=20

ERROR 2014-07-10 10:51:55,690 (Agents thread) - Exception tossed: Service 'A&= #39; of type 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is alread= y active

org.apache.manifoldcf.cor= e.interfaces.ManifoldCFException: Service 'A' of type 'AGENT_org.apache.manifoldcf.crawler.system= .CrawlerAgent' is already active


Regards.




<= div>


=C2=A0
<= /div>


On= Thu, Jul 10, 2014 at 10:12 AM, Karl Wright <daddywri@gmail.com>= wrote:
bq. So the only differ= ence i could observe is DB which is HSQL vs Postgres, rest everything is sa= me.

Right, exactly.=C2=A0 Only this time we have some certainty that = the configuration is otherwise correct for synchronization and processes.
The next step is the following:

- START with th= e same multiprocess-file-example directory and processes you verified yeste= rday
- Stop all MCF processes/web applications that are using that= multiprocess-file-example properties file:
=C2=A0 - Either stop the Tomcat process, or better yet go to the &quo= t;admin" tomcat application and stop all three MCF web applications
=C2=A0 - Go to the start-agents.sh window and ^C that process
=C2=A0 - Go to the start-database.sh window and ^C that process
- Change the properties.xml file under multiprocess-file-example to p= oint to the PostgresDB instance you want it to use.=C2=A0 This instance CAN= NOT be in use by any other MCF cluster or process.
=C2=A0 - Replac= e the following THREE properties in properties.xml with your PostgreSQL equ= ivalent, as follows:

>>>>>>
=C2=A0 <property name=3D"org.apache= .manifoldcf.databaseimplementationclass" value=3D"org.apache.mani= foldcf.core.database.DBInterfacePostgreSQL"/>
=C2=A0 <propert= y name=3D"org.apache.manifoldcf.dbsuperusername" value=3D"&l= t;something>"/>
=C2=A0 <property name=3D"org.apache.manifoldcf.dbsuperuserpassword&= quot; value=3D"<something>"/>
<<<<<<= ;

You may also choose to set these properties, to avoid collis= ion with other instances.=C2=A0 You can use whatever value you like for the= se, within what PostgreSQL allows:

org.apache.manifoldcf.database.name
org.apache.manifoldcf.database= .username
org.apache.manifoldcf.database.password

- R= un the initialize script:
=C2=A0 - cd multiprocess-file-example ; initialize.sh .=C2=A0 If this= works, you have the database configured properly.=C2=A0 If it fails, look = at the parameters you provided and figure out what is wrong before proceedi= ng

- start the ManifoldCF processes and web applications:
=C2=A0 - cd= multiprocess-file-example ; start-agents.sh
=C2=A0 - go to the to= mcat instance, and start it, or start all the web applications

- create connections and jobs, and try a crawl.=C2=A0 Let me know how it w= orks.

Karl



On Thu, Jul = 10, 2014 at 12:25 AM, lalit jangra <lalit.j.jangra@gmail.com>= ; wrote:
Thanks = a lot Karl,

Yes, it works perfectly as you suggested in last m= ail.

So the only difference i could observe is DB which is HSQL vs Pos= tgres, rest everything is same.

Please suggest.

Regards.


=


On Wed, Jul 9, 2014 at 11:35 PM, Karl Wright <= ;daddywri@gmail.com= > wrote:
Hi Lalit,


<= /div>
Sorry, I thought my instructions were clear.=C2=A0 For this exerc= ise, I want you to use ONLY the example under dist/multiprocess-file-exampl= e.=C2=A0 DO NOT use the example under dist/example.

Here are the instructions again, clarified and fleshed out:<= div>

- Start with a fresh build of MCF
- Go to the multiprocess-f= ile example and start that in Jetty, with NO changes:
=C2=A0 - cd multiprocess-file-example
=C2=A0 - start-database.sh
=C2=A0 - initialize.sh<= br>
=C2=A0 - start-agents.sh
=C2=A0 - in a separate= window: cd multiprocess-file-example ; start-webapps.sh
=C2= =A0 - create connections and job, see if it runs
- If that doesn't work, open a ticket.=C2=A0 If that DOES work, proceed= as follows:
- Shut down the webapps process:
=C2=A0= - go to the window where start-webapps.sh is running, and type ^C
- Set up a fresh tomcat instance
- Configure tomcat to point= to the multiprocess-file example properties.xml:
=C2=A0 - Mo= dify the tomcat startup scripts to include -Dorg.apache.= manifoldcf.configfile=3D<path_to_dist/multiprocess-file-example/properti= es.xml>
=C2=A0 - Start the new instance of tomcat
- Deploy = the wars to that tomcat (except combined war)
=C2=A0 - = Use the Tomcat admin tool to deploy mcf-crawler-ui.war, mcf-authority-servi= ce.war, and mcf-api-service.war
- Go to the UI as served by Tomcat.=C2=A0 You should see your co= nnections, job, etc just as before.=C2=A0 Run the job.=C2=A0 Does it work, = yes or no?


Thanks,
Karl



=




On Wed, Jul 9, 2014 at 5:58 PM, lalit jangra <lal= it.j.jangra@gmail.com> wrote:
Thanks Karl,

Indeed i really appreciate your = help and promptness.

I tried the way you suggest and below are results.

1= . Took a fresh MCF, started it using start.jar under dist/example using jet= ty.
2. Under multiprocess-file-example, initialized DB & started agen= t using .start-agents.sh.
3. Created all connection and jobs. Star= ted job & it worked fine.
4. Next i shutdown jetty as started = in step1.
5. Setup a fresh tomcat and pointed to multiprocess-file-example/prop= erties.xml in its setenv.sh using MCF_HOME.
6. Started tomcat &= ; deployed all but combined war.
7. Accessed mcf-crawler-ui war &a= mp; accessed job.
8. On Job tab i get connection refused error.
9. I= f i start DB using ./start-databse.sh, i get "Error getting connection= :invalid authorization exception-not found:manifoldcf" message.

Really embarrassing for me as its not working.

=
Regards.

On Wed, Jul 9, 2014 at 6:52 PM, Karl Wright= <daddywri@gmail.com> wrote:
Hi Lalit,

Since I'm quite busy at work today, can you do the = following:

- Start with a fresh build of MCF
- Go to the multiproce= ss-file example and start that in Jetty, with NO changes
- If that doesn't work, open a ticket.=C2=A0 If that DOES work, p= roceed as follows:
- Shut down the webapps process in the example<= br>
- Set up a fresh tomcat instance
- Configure tomcat = to point to the multiprocess-file example properties.xml.
- Deploy the wars to that tomcat (except combined war)
- Once agai= n, try your job.=C2=A0 Let me know the result.

Thanks,
Karl


On Wed, Jul 9, 2014 at 1:47 PM, Karl Wright <d= addywri@gmail.com> wrote:
Hi lalit,

I don't see anything wrong with your environment.=C2=A0 I suspect your= configuration is what is wrong.

Karl

On Wed, Jul 9, 2014 at 1:46 PM, lalit jangra <lalit.j.jangra@gmail= .com> wrote:
Thanks Karl,

I agree with you. I am using Red = Hat 6.4 with Postgres 9.2 & tomcat 7.

Can this be an issue ?
=
Regards.


On Wed, Jul 9, 2014 at 6:43 PM, Karl Wright <daddywri@gmail.com>= wrote:
Hi Lalit,

I tried the zookeeper example = here this morning and it worked perfectly.=C2=A0 I am happy to try the file= system example too, when I have a moment, but I very much doubt there is a= ny problem with that code specifically.

Remember that this is NOT a database issue.=C2=A0 It is a synchronizati= on issue ONLY.=C2=A0 So it does not matter what database instance you point= it to.

Karl


On Wed, Jul 9, 2014 at 1:32 PM, lalit jangra <lalit.j.jangra@gmail= .com> wrote:
Thanks Karl,

I did it as you mentioned but sti= ll getting same error. This time also i setup new DB and tried. Also i coul= d see syncharea folder empty after lock cleaning.

Regards.


On Wed, Jul 9, 2014 at 6:21 PM, Karl Wright <daddywri@gmail.com>= ; wrote:
Hi Lalit,

If you were using the multiprocess-file-example before, and you did not sh= ut things down cleanly, you should do: ./lock-clean.sh first.

Karl



On Wed, Jul 9, 2014 at 1:10 PM, lalit ja= ngra <lalit.j.jangra@gmail.com> wrote:
Hi Karl,
<= br>I think its my bad luck today. :(

I tried the way you just= suggested but still getting same error and job not able to run. What i did= .

1. Created a brand new DB schema using ./executecommand.sh org.ap= ache.manifoldcf.core.DBCreate username password
2. Initialized DB = schema using ./initialize.sh
3. Start agent process A using ./sta= rt-agents.sh
4. Updated tomcat/bin/setenv.sh file to add "-Dorg.apache.manifo= ldcf.configfile" parameter to point to properties.xml under dist/multi= process-file-example.
5. Started Tomcat.
6. Deployed all = wars except combined war.
7. Started job

Job still not able to take off with same = error in logs.

ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exce= ption tossed: =C2=A0Service 'A' of type =C2=A0'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is = already
active

I think its not my day = today.

Regards.

<= div>

On Wed, Jul 9, 2014 at 4:02 PM, Karl Wright <daddywri@gmail.com>= wrote:
Hi Lalit,

= In theory the combined war will run with external locking, it's just th= at nobody ever thought that would be how people would use it, and therefore= it hasn't been tested etc.

I would suggest that for a development setup, you use the Jetty s= imple example.=C2=A0 For a production setup, you should adopt the standard = distributed arrangement as follows:

- a deployed mcf-crawler-u= i war
- a deployed mcf-authority-service war
- an (optional) deplo= yed mcf-api-service war
- one or more agents processes

This is how it is done in both kinds of multiprocess examples, and is des= cribed thoroughly in the how-to-build-and-deploy document online.

Thanks,
Karl



On Wed, Jul 9, 2014 at 10:56 AM, lalit jangra &= lt;lalit.j.ja= ngra@gmail.com> wrote:
Tha= nks Karl,

My idea was as its name suggests mcf-combined-w= ar as i was trying to take advantage of agent process inside war. My assump= tion is wrong here as i can see here.

In this case, which war should i use - mcf-crawler-ui.war? If i use th= is or any of other wars, then should i start agent process?

Please s= uggest.

Regards.


On Wed, Jul 9, 2014 at 3:48 PM, Karl Wri= ght <daddywri@gmail.com> wrote:
Hi Lalit,

Probably it is not a database = issue, but how locking is configured.=C2=A0 Typically the combined war is r= un in a non-clustered environment with no external synchronization required= (as it is in the example I pointed you at).=C2=A0 We don't currently h= ave an example that uses combined war with external locking -- that would n= ot make much sense.=C2=A0 Can you tell me why you are trying to do it that = way?

Karl


On Wed, Jul 9, 2014 at 10:46 AM, lalit jan= gra <lalit.j.jangra@gmail.com> wrote:
Hi Karl,

if i try ./start-combined.sh, the job runs and= i could see it finishing gracefully with updating index in solr.

I assume there is some DB issue here? Any suggestions?

Regards.


On Wed, Jul 9, 2014 at 3:17 PM, Karl Wright <dad= dywri@gmail.com> wrote:
Hi Lalit,

Please do the following: In a= n unmodified dist/example directory, run the start-combined script.=C2=A0 O= pen the UI, create the connections, and try to run the job.=C2=A0 Does it w= ork for you?


K= arl



On Wed, Jul 9, 2014 at 10:09 AM, lalit jangra <lalit.j.jangra@gmail= .com> wrote:
Thanks Karl,

I have reconfigured everything and this ti= me i am starting tomcat and deploying mcf-combined-war. I created output co= nnection, repository connection & job. Still when i start my job, it do= es not go ahead and stuck at starting up status.


Regards.



On Wed, Jul 9, 2014 at 12:49 PM, Ka= rl Wright <daddywri@gmail.com> wrote:
Hi = Lalit,

You do not use combined.war AND an agents process at th= e same time.=C2=A0 The combined war INCLUDES the agents process.

Karl


On Wed, Jul 9, 2014 at 6:18 AM, lalit jangra <lalit.j.jangra@gmail= .com> wrote:
Hi Karl,

As per your sugg= estion, i built a new MCF 1.5.1 instance, con from source , configured it a= nd pointed to local postgresql DB in properties.xml=C2=A0 under /multiproce= ss-file-example. Then i tool a brand new tomcat and configured for MCF in i= t in setenv.sh for properties.xml using "-Dorg.apache.manifoldcf.confi= gfile" argument and deployed mcf-combined.war using app manager.

First i initialized DB using ./initialize.sh command & the mo= ment i started agent using ./start-agents.sh command under /multiprocess-fi= le-example, i got the error, even when tomcat is not started at this time. = I created a job which never runs & get stuck.


ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exception tossed: =C2=A0Service 'A' of type =C2=A0'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is = already
=C2=A0 active
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Service 'A&#= 39; =C2=A0of type 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent&= #39; is =C2=A0already active

But if i try to run jetty bundle = using start.jar under example, no such error but it does not fulfil my requ= irements.

Regards.


On Wed, Jul 9, 2014 at 10:03 AM, Karl Wright <daddywri@gmail.com>= wrote:
So, lalit, if you run the multiprocess example without any changes, do
you see this? =C2=A0I don't.

Karl

Sent from my Windows Phone

-----Original Message-----
From: lalit jangra
Sent: 7/9/2014 12:17 AM
To: Karl Wright
Cc: user@ma= nifoldcf.apache.org
Subject: Re: Apache ManifoldCF job stuck up





Thanks Karl,


I am currently running only single agent process on single machine
without clustering. I have two environments and i could see this issue
coming up at both places.

While trying to start agent, i could see below error. When i am trying
to start agent, it says below error and exits. But there is no agent
process already running.


[root@server1 multiprocess-file-example]# ./start-agents.sh &

[1] 5020

[root@server1 multiprocess-file-example]# Running...

Configuration file successfully read

org.apache.manifoldcf.core.interfaces.ManifoldCFException: Service 'A&#= 39;
of type 'AGENT' is already active

=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.core.lockmanager.BaseL= ockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.core.lockmanager.BaseL= ockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.core.lockmanager.LockM= anager.registerServiceBeginServiceActivity(LockManager.java:69)

=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.agents.AgentRun.doExec= ute(AgentRun.java:54)

=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.agents.BaseAgentsIniti= alizationCommand.execute(BaseAgentsInitializationCommand.java:37)

=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.agents.AgentRun.main(A= gentRun.java:93)



[1]+ =C2=A0Exit 1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0./start-agents.sh


If i am able to successfully start agent using ./start-agents.sh,
still it throws same error but i have no other process running.

Regards.




On Tue, Jul 8, 2014 at 2:38 PM, Karl Wright <daddywri@gmail.com> wrote:

Hi lalit,

This occurs when you have more than one agents process with the same
=C2=A0process id using the same shared file system directory / zookeeper =C2=A0cluster. =C2=A0There is no other way it can occur.

Thanks,
=C2=A0Karl

Sent from my Windows Phone



-----Original Message-----
=C2=A0From: lalit jangra
=C2=A0Sent: 7/8/2014 8:38 AM
=C2=A0To: u= ser@manifoldcf.apache.org
=C2=A0Subject: Re: Apache ManifoldCF job stuck up



Thanks Karl,


I have tried steps you suggested and it worked on one instance.

But on another instance i am still not able to resolve this issue.
=C2=A0Along with steps you mentioned, i tried recreating DB instance again,=
=C2=A0setting up new MCF instance , cleaning locks and then starting agents=
=C2=A0first & then tomcat. But issue still persists.

If i try to run ./start-agents.sh, i get this error for agent A.


ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exception tossed:
=C2=A0Service 'A' of type
=C2=A0'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is = already
=C2=A0active

org.apache.manifoldcf.core.interfaces.ManifoldCFException: Service 'A&#= 39;
=C2=A0of type 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent&= #39; is
=C2=A0already active

=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.core.lockmanager.BaseL= ockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.core.lockmanager.BaseL= ockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.core.lockmanager.LockM= anager.registerServiceBeginServiceActivity(LockManager.java:69)

=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.agents.system.AgentsDa= emon.checkAgents(AgentsDaemon.java:270)




But if i try to run ./start-agents-2.sh, i see similar error but agent B




ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exception tossed:
=C2=A0Service 'B' of type
=C2=A0'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is = already
=C2=A0active

org.apache.manifoldcf.core.interfaces.ManifoldCFException: Service 'B&#= 39;
=C2=A0of type 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent&= #39; is
=C2=A0already active

=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.core.lockmanager.BaseL= ockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.core.lockmanager.BaseL= ockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.core.lockmanager.LockM= anager.registerServiceBeginServiceActivity(LockManager.java:69)

=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.agents.system.AgentsDa= emon.checkAgents(AgentsDaemon.java:270)




Regards.





On Mon, Jul 7, 2014 at 4:55 PM, Karl Wright <daddywri@gmail.com> wrote:




Hi Lalit,


If you are using file synchronization, you cannot expect MCF to clean
=C2=A0up itself unless you shut it down cleanly. =C2=A0You should be either= using
=C2=A0^C or plain kill, NEVER kill -9. =C2=A0kill -9 will leave dangling lo= cks.



To clean up dangling locks:



- shut ALL manifoldcf processes and web apps down


- run the lock-clean script


- start up the processes again



=C2=A0Zookeeper synchronization, by the way, does not have this kind of pro= blem.

Thanks,
=C2=A0Karl








On Mon, Jul 7, 2014 at 11:49 AM, lalit jangra <lalit.j.jangra@gmail.com> wrote= :




Hi,


I configured MCF 1.5.1 to run with Postgresql DB and tomcat 7.
=C2=A0Initially i created all connections, created a an alfresco job and it=
=C2=A0all worked fine.


Next for updates, i stopped, tomcat and agent process running. Then i
=C2=A0updated CmisRepositoryConnector.java with my own code and run "a= nt
=C2=A0build" at root of MCF. It updated all code and jars file.



Also properties.xml is reset under /dist/multiprocess-file/example
=C2=A0which i updated to connect to Postgresql DB and logging
=C2=A0configurations.



I started tomcat then agent process. =C2=A0Finally i started job to crawl =C2=A0alfresco but it got stuck and moving on. I checked into
=C2=A0/dist/multiprocess-file/example/logs/manifoldcf.log file and could se= e
=C2=A0below error.


ERROR 2014-07-07 16:09:04,936 (Agents thread) - Exception tossed:
=C2=A0Service '' of type
=C2=A0'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is = already
=C2=A0active

org.apache.manifoldcf.core.interfaces.ManifoldCFException: Service '= 9;
=C2=A0of type 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent&= #39; is
=C2=A0already active

=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.core.lockmanager.BaseL= ockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.core.lockmanager.BaseL= ockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.core.lockmanager.LockM= anager.registerServiceBeginServiceActivity(LockManager.java:69)

=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.agents.system.AgentsDa= emon.checkAgents(AgentsDaemon.java:270)

=C2=A0 =C2=A0 =C2=A0 =C2=A0 at org.apache.manifoldcf.agents.system.AgentsDa= emon$AgentsThread.run(AgentsDaemon.java:208)









I created another job but that got stuck too.Is DB got corrupt due to rebui= ld?




Also is this the right way to build MCF (I hope its correct way). Now
=C2=A0what should i do to fix this issue?





Please help.

Regards,
=C2=A0Lalit Jangra.




--
=C2=A0Regards,
=C2=A0Lalit Jangra.




--
Regards,
Lalit Jangra.



<= font color=3D"#888888">--
Regards,
Lalit Jangra.

<= /span>
=


--
Regards,
Lalit Jangra.




= --
Regards,
Lalit Jangra.




= --
Regards,
Lalit Jangra.




= --
Regards,
Lalit Jangra.




= --
Regards,
Lalit Jangra.




= --
Regards,
Lalit Jangra.




= --
Regards,
Lalit Jangra.



= --
Regards,
Lalit Jangra.



--
Regards,Lalit Jangra. --047d7b3a89787715b504fdd3e2b9--