oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Bennett <lmzxq....@gmail.com>
Subject Re: crawler actionIds
Date Thu, 22 Mar 2012 21:52:06 GMT
Hey Brian,

> okay... got a patch up: https://issues.apache.org/jira/browse/OODT-428

Thanks! I'll try it out tomorrow morning :-)

> patch it into your crawler an let me know if it solves your problem

Will do!

> Also, on another note... that fix you did for *Metadata* to *MetadataFile* CrawlerAction...
was it for: MoveMetadataToFailureDir -> MoveMetadataFileToFailureDir... or did i just find
a new typo of mine again?

Sorry did not commit that change yet. But it is for MoveMetadataToFailureDir -> MoveMetadataFileToFailureDir

Cheers,
Tom

> -brian
> 
> On Mar 22, 2012, at 01:36 PM, Brian Foster <holenoter@me.com> wrote:
> 
>> hey thomas,
>> 
>> i know what the problem is... working on the fix right now... i'll create an issue
describing the problem and post the fix so you can patch it in as soon as i have something.
>> 
>> -brian
>> 
>> On Mar 22, 2012, at 07:07 AM, Thomas Bennett <lmzxq.tom@gmail.com> wrote:
>> 
>>> Sorry forgot to mention that I'm using 0.4. ;-)
>>> 
>>> On 22 March 2012 16:02, Thomas Bennett <lmzxq.tom@gmail.com> wrote:
>>> 
>>> > Hi,
>>> >
>>> > I can't seem to add multiple actionIds when I launch a crawler. Specifying
>>> > a single actionId seems to work.
>>> >
>>> > I'm just wondering if I need to specify the actionId's in a different way.
>>> > Note I'm not using the script + abs paths to configuration files.
>>> >
>>> > Cheers,
>>> > Tom
>>> >
>>> > Command executed:
>>> >
>>> > ~$ java -Dorg.apache.oodt.cas.cli.debug=true
>>> > -Djava.util.logging.config.file=/var/kat/katconfig/static/oodt/etc/logging.properties
>>> > -Djava.ext.dirs=/usr/local/oodt/cas-crawler/lib
>>> > -Dorg.apache.oodt.cas.crawl.bean.repo=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/crawler-config.xml
>>> > -Dorg.apache.oodt.cas.cli.action.spring.config=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/cmd-line-actions.xml
>>> > -Dorg.apache.oodt.cas.cli.option.spring.config=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/cmd-line-options.xml
>>> > org.apache.oodt.cas.crawl.CrawlerLauncher --operation --launchAutoCrawler
>>> > --productPath /var/kat/data --filemgrUrl http://127.0.0.1:9101--mimeExtractorRepo
>>> > /var/kat/katconfig/static/oodt/cas-crawler/policy/mime-extractor-map.xml
>>> > --clientTransferer
>>> > org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory
>>> > --failureDir /var/kat/failed/ --actionIds DeleteDataFile DeleteMetadataFile
>>> >
>>> > Error message:
>>> > Setting property 'AutoDetectProductCrawler.mimeExtractorRepo'
>>> > Setting property 'StdProductCrawler.clientTransferer'
>>> > Setting property 'MetExtractorProductCrawler.clientTransferer'
>>> > Setting property 'AutoDetectProductCrawler.clientTransferer'
>>> > Setting property 'StdProductCrawler.filemgrUrl'
>>> > Setting property 'MetExtractorProductCrawler.filemgrUrl'
>>> > Setting property 'AutoDetectProductCrawler.filemgrUrl'
>>> > Setting property 'StdProductCrawler.actionIds'
>>> > Setting property 'MetExtractorProductCrawler.actionIds'
>>> > Setting property 'AutoDetectProductCrawler.actionIds'
>>> > Setting property 'StdProductCrawler.productPath'
>>> > Setting property 'MetExtractorProductCrawler.productPath'
>>> > Setting property 'AutoDetectProductCrawler.productPath'
>>> > Setting property 'MoveMetadataFileToFailureDir.toDir'
>>> > Setting property 'MovePushpullMetFileToFailureDir.toDir'
>>> > Setting property 'MoveDataFileToFailureDir.toDir'
>>> > Exception in thread "main" java.lang.RuntimeException:
>>> > org.apache.oodt.cas.cli.exception.CmdLineActionException: Failed to launch
>>> > crawler : No bean named 'DeleteDataFile,DeleteMetadataFile' is defined
>>> > at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:191)
>>> > at org.apache.oodt.cas.crawl.CrawlerLauncher.main(CrawlerLauncher.java:36)
>>> > Caused by: org.apache.oodt.cas.cli.exception.CmdLineActionException:
>>> > Failed to launch crawler : No bean named
>>> > 'DeleteDataFile,DeleteMetadataFile' is defined
>>> > at
>>> > org.apache.oodt.cas.crawl.cli.action.CrawlerLauncherCliAction.execute(CrawlerLauncherCliAction.java:61)
>>> > at
>>> > org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
>>> > at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
>>> > ... 1 more
>>> > Caused by:
>>> > org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean
>>> > named 'DeleteDataFile,DeleteMetadataFile' is defined
>>> > at
>>> > org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanDefinition(DefaultListableBeanFactory.java:387)
>>> > at
>>> > org.springframework.beans.factory.support.AbstractBeanFactory.getMergedLocalBeanDefinition(AbstractBeanFactory.java:968)
>>> > at
>>> > org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:246)
>>> > at
>>> > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
>>> > at
>>> > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:168)
>>> > at
>>> > org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:885)
>>> > at
>>> > org.apache.oodt.cas.crawl.action.CrawlerActionRepo.loadActionsFromBeanFactory(CrawlerActionRepo.java:71)
>>> > at org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:84)
>>> > at org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:76)
>>> > at
>>> > org.apache.oodt.cas.crawl.cli.action.CrawlerLauncherCliAction.execute(CrawlerLauncherCliAction.java:58)
>>> > ... 3 more
>>> >

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message