oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (388J)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: Problem when running pushpull
Date Thu, 02 Aug 2012 02:42:53 GMT
Hi YunHee,

Can you specifically tell me what's missing on the OODT file manager
start up guide?

Have you checked out our wiki?

https://cwiki.apache.org/confluence/display/OODT/Home

Try the documentation there, including the updated FM user guide:

https://cwiki.apache.org/confluence/display/OODT/OODT+Filemgr+User+Guide

Cheers,
Chris

On Aug 1, 2012, at 4:20 PM, YunHee Kang wrote:

> Hi Chris,
> 
> I fullly made sense what you mentioned.
> The root of this problem  occurred by this "Catalog exception" is
> caused by insufficient configuration of my filemgr.
> To be honest, my file manager has not been built yet.
> But it is difficult for me to configure properties of my file manager
> depending on oodt homepage.
> I think what I need is some materials including examples of
> configuration and test codes for them.
> Let me know how to seamlessly connect two components, a file manager
> and a pushpull.
> 
> Thanks,
> Yunhee
> 
> 2012/8/2 Mattmann, Chris A (388J) <chris.a.mattmann@jpl.nasa.gov>:
>> Hey YunHee,
>> 
>> On Aug 1, 2012, at 4:17 AM, YunHee Kang wrote:
>> 
>>> Hi Brian,
>>> 
>>> 
>>> When I was reading a file named "push_pull_framework.properties", I
>>> understood  how  a metadata file is named.
>>> #extension for temporary metadata file
>>> org.apache.oodt.cas.pushpull.write.met.file=true
>>> org.apache.oodt.cas.pushpull.met.file.extension=info.tmp
>>> 
>>> And I found a metadata file written in XML  having its name ended with
>>> ".info.tmp"  in the staging area, which  was generated  by pushpull
>>> script.
>>> 
>>> Hence I tried to catalog the downloaded file and its metadata file in
>>> the staging area by filemgr-client and I am sure that  the file was
>>> ingested by filemgr:
>>> 
>>> yhkang@yunhee-kisti:~/oodt-0.5/cas-filemgr/bin$ ./filemgr-client-tes.sh
>>> Aug 1, 2012 7:37:37 PM
>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient <init>
>>> INFO: Loading File Manager Configuration Properties from:
>>> [../etc/filemgr.properties]
>>> Aug 1, 2012 7:37:38 PM
>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>> INFO: Running Met Extractor:
>>> [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for
>>> product type: [GenericFile]
>>> Aug 1, 2012 7:37:38 PM
>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>> INFO: Running Met Extractor:
>>> [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor]
>>> for product type: [GenericFile]
>>> Aug 1, 2012 7:37:38 PM
>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>> INFO: Running Met Extractor:
>>> [org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocationExtractor]
>>> for product type: [GenericFile]
>>> Aug 1, 2012 7:37:38 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog toDoc
>>> WARNING: No Metadata specified for product [tesProduct] for required
>>> field [DataVersion]: Attempting to continue processing metadata
>>> Aug 1, 2012 7:37:38 PM
>>> org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferer moveFile
>>> INFO: LocalDataTransfer: Moving File:
>>> file:///home/yhkang/oodt-0.5/cas-filemgr/bin/TES-Aura_L2-CO2-Nadir_r0000002147_F06_09.he5
>>> to file:/home/yhkang/files/tesProduct/TES-Aura_L2-CO2-Nadir_r0000002147_F06_09.he5
>>> Aug 1, 2012 7:37:38 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog toDoc
>>> WARNING: No Metadata specified for product [tesProduct] for required
>>> field [DataVersion]: Attempting to continue processing metadata
>>> ingestProduct: Result: e9f22900-dbc4-11e1-a524-6bffd22dafd0
>>> 
>>> I also checked up  the file indigested by query_tool.
>>> ./query_tool --url http://localhost:8000 --lucene -query
>>> CAS.ProductName:tesProduct
>>> e9f22900-dbc4-11e1-a524-6bffd22dafd0
>>> 
>>> I am wondering why the "catalog exception" was occurred as pushpull
>>> was running even if my file manager was working well .
>>> Could you recommend me any material to understand mechanism between
>>> filemgr and pushpull.
>> 
>> Hi YunHee,
>> 
>> This is probably a chicken and egg issue -- if your file manager catalog hasn't
>> been built yet and pushpull checks product uniquess the catalog won't exist
>> and you'll get the CatalogException the first time it's called. All times after that,
>> the catalog will exist and you won't get it.
>> 
>> Make sense?
>> 
>> Cheers,
>> Chris
>> 
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW:   http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> 


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


Mime
View raw message