oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Foster <holeno...@mac.com>
Subject RE: cas-crawler wierdness
Date Wed, 14 Mar 2012 21:38:55 GMT
hey tom,

ya Review Board isn't intuitive at all... took me like 30 mins of trial and error to figure
it all out... basically after you click "New Review Request" you pick "oodt" from the "Repository"
drop down, then for "Base Directory" put a relative path (relative to oodt svn base directory: https://svn.apache.org/repos/asf/oodt/)...
so say you are creating a crawler patch your path would be "trunk/crawler"... then just upload
your patch and all should work... in your case though your patch is relative to the trunk
so just put "trunk" for "Base Directory"

-brian

On Mar 14, 2012, at 06:18 AM, Thomas Bennett <lmzxq.tom@gmail.com> wrote:

Hi Brian + Chris,

Thanks, for you're help! All sorted.

I attempted to try out Review Board, I was unable to load a diff, it kept on rejecting my
diff files.

I've attached the diff (to update logging from WARNING to SEVERE) to the JIRA issue.

For #2, what chris said about the file://** makes sense... try that and let me know how that
works ou

Yup that did the trick, although I needed three slashes file:/// :)

I also had to modify the constructor-arg values in cmd-line-options.xml. 

<constructor-arg value="file:///var/kat/katconfig/static/oodt/cas-crawler/policy/precondition-beans.xml"
/> 

All working fine now except when I execute:

$ java -Djava.util.logging.config.file=/var/kat/katconfig/static/oodt/etc/logging.properties
-Djava.ext.dirs=/usr/local/oodt/cas-crawler/lib -Dorg.apache.oodt.cas.crawl.bean.repo=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/crawler-config.xml
 -Dorg.apache.oodt.cas.cli.action.spring.config=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/cmd-line-actions.xml
-Dorg.apache.oodt.cas.cli.option.spring.config=file:///var/kat/katconfig/static/oodt/cas-crawler/policy/cmd-line-options.xml
org.apache.oodt.cas.crawl.CrawlerLauncher --operation --launchAutoCrawler --productPath /var/kat/data
--filemgrUrl http://127.0.0.1:9101 --mimeExtractorRepo /var/kat/katconfig/static/oodt/cas-crawler/policy/mime-extractor-map.xml
--failureDir /var/kat/failed

The output looks like:
Setting property 'AutoDetectProductCrawler.mimeExtractorRepo'
Setting property 'StdProductCrawler.productPath'
Setting property 'MetExtractorProductCrawler.productPath'
Setting property 'AutoDetectProductCrawler.productPath'
Setting property 'MoveMetadataToFailureDir.toDir'
Setting property 'MovePushpullMetFileToFailureDir.toDir'
Setting property 'MoveDataFileToFailureDir.toDir'
Setting property 'StdProductCrawler.filemgrUrl'
Setting property 'MetExtractorProductCrawler.filemgrUrl'
Setting property 'AutoDetectProductCrawler.filemgrUrl'
ERROR: Could not process key 'MoveMetadataToFailureDir.toDir' in PropertyOverrideConfigurer;
nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean
named 'MoveMetadataToFailureDir' is defined

I'm a bit uncertain how to properly define failed to ingest actions with the new interface.

I see when calling help:

 -fd,    --failureDir <directory>                           Directory
where files will be moved on failure
                                                           
  Requirement Rules: 
                                                           
   [launchStdCrawler : OPTIONAL, launchMetCrawler : 
                                                           
   OPTIONAL, launchAutoCrawler : OPTIONAL] 

                                                           
  Handler: 
                                                           
   Set the following bean properties: 
                                                           
   [MoveMetadataToFailureDir.toDir, 
                                                           
   MovePushpullMetFileToFailureDir.toDir, 
                                                           
   MoveDataFileToFailureDir.toDir] 

How do I set the bean properties from the command line :)

I see there are two definitions - MoveMetadataFileToFailureDir and MoveMetadataToFailureDir,
should these be the same?

Cheers,
Tom
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
    • Unnamed multipart/related (inline, None, 0 bytes)
View raw message