manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Importing old configuration data
Date Mon, 23 Apr 2012 17:33:38 GMT
The problem is not with the current version; it's with the ManifoldCF
0.3 version, which did not output things in the correct order.
Therefore the outputs do not get processed before the jobs do, leading
to this problem.

There are two possible solutions.  The first is to hack up your copy
of 0.5 to do TWO passes over the data file, looking for "outputs" on
the first pass, and looking for everything on the second.  The second
is create the "Solr" output connection by hand; running the script
after that should populate everything.

Thanks,
Karl


On Mon, Apr 23, 2012 at 9:25 AM, Karl Wright <daddywri@gmail.com> wrote:
> Hmm.  Your output shows that the output connector is indeed registered:
>
>  ./processes/script/executecommand.sh
> org.apache.manifoldcf.agents.RegisterOutput
> org.apache.manifoldcf.agents.output.solr.SolrConnector SolrConnector
>
> If you have no passwords in the config, you can send me the file and
> I'll debug it to see what's going on.
>
> Karl
>
> On Mon, Apr 23, 2012 at 9:04 AM, Karl Wright <daddywri@gmail.com> wrote:
>> Hmm.  There have been no changes to the format or meaning of the
>> configuration file AFAIK for quite a while.
>>
>> This is complaining about there not being an output connection called
>> "Solr" in place at the time it tries to create a job.  In theory, on
>> import, the creation of connections takes place before the creation of
>> jobs.  Is there any possibility that the connectors have not been
>> registered?  I think they would need  to be before you attempt the
>> import.
>>
>> Karl
>>
>> On Mon, Apr 23, 2012 at 8:57 AM, Erlend Garåsen <e.f.garasen@usit.uio.no>
wrote:
>>>
>>> I'm trying to import my old configuration files into my freshly installed
>>> MCF version, but I'm afraid that these configuration data is too old (from
>>> version 0.3, I think). I get the following error: "Database exception:
>>> Exception doing query: ERROR: insert or update on table "jobs" violates
>>> foreign key constraint "jobs_outputname_fkey"
>>>  Detail: Key (outputname)=(Solr) is not present in table
>>> "outputconnections"".
>>>
>>> Here's what I tried to do. The last step fails:
>>> ./processes/script/executecommand.sh org.apache.manifoldcf.agents.Install
>>>  ./processes/script/executecommand.sh
>>> org.apache.manifoldcf.agents.RegisterOutput
>>> org.apache.manifoldcf.agents.output.solr.SolrConnector SolrConnector
>>> ./processes/script/executecommand.sh org.apache.manifoldcf.agents.Register
>>> org.apache.manifoldcf.crawler.system.CrawlerAgent
>>> ./processes/script/executecommand.sh org.apache.manifoldcf.crawler.Register
>>> org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector
>>> WebCrawler
>>> ./processes/script/executecommand.sh
>>> org.apache.manifoldcf.crawler.ImportConfiguration export_mcf.zip
>>>
>>> Complete stack trace:
>>>
>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Database
>>> exception: Exception doing query: ERROR: insert or update on table "jobs"
>>> violates foreign key constraint "jobs_outputname_fkey"
>>>  Detail: Key (outputname)=(Solr) is not present in table
>>> "outputconnections".
>>>        at
>>> org.apache.manifoldcf.core.database.Database.executeViaThread(Database.java:680)
>>>        at
>>> org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:708)
>>>        at
>>> org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1393)
>>>        at
>>> org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
>>>        at
>>> org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:185)
>>>        at
>>> org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
>>>        at
>>> org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
>>>        at
>>> org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
>>>        at org.apache.manifoldcf.crawler.jobs.Jobs.save(Jobs.java:756)
>>>        at
>>> org.apache.manifoldcf.crawler.jobs.JobManager.save(JobManager.java:576)
>>>        at
>>> org.apache.manifoldcf.crawler.jobs.JobManager.importConfiguration(JobManager.java:230)
>>>        at
>>> org.apache.manifoldcf.crawler.system.ManifoldCF.importConfiguration(ManifoldCF.java:1100)
>>>        at
>>> org.apache.manifoldcf.crawler.ImportConfiguration.doExecute(ImportConfiguration.java:40)
>>>        at
>>> org.apache.manifoldcf.crawler.BaseCrawlerInitializationCommand.execute(BaseCrawlerInitializationCommand.java:37)
>>>        at
>>> org.apache.manifoldcf.crawler.ImportConfiguration.main(ImportConfiguration.java:57)
>>> Caused by: org.postgresql.util.PSQLException: ERROR: insert or update on
>>> table "jobs" violates foreign key constraint "jobs_outputname_fkey"
>>>  Detail: Key (outputname)=(Solr) is not present in table
>>> "outputconnections".
>>>        at
>>> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2103)
>>>        at
>>> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1836)
>>>        at
>>> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
>>>        at
>>> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:512)
>>>        at
>>> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:388)
>>>        at
>>> org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:334)
>>>        at
>>> org.apache.manifoldcf.core.database.Database.execute(Database.java:839)
>>>        at
>>> org.apache.manifoldcf.core.database.Database$ExecuteQueryThread.run(Database.java:640)
>>>
>>>
>>> --
>>> Erlend Garåsen
>>> Center for Information Technology Services
>>> University of Oslo
>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050

Mime
View raw message