oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (388J)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: Problem when running pushpull
Date Wed, 01 Aug 2012 17:56:25 GMT
Hey YunHee,

On Aug 1, 2012, at 4:17 AM, YunHee Kang wrote:

> Hi Brian,
> 
> 
> When I was reading a file named "push_pull_framework.properties", I
> understood  how  a metadata file is named.
> #extension for temporary metadata file
> org.apache.oodt.cas.pushpull.write.met.file=true
> org.apache.oodt.cas.pushpull.met.file.extension=info.tmp
> 
> And I found a metadata file written in XML  having its name ended with
> ".info.tmp"  in the staging area, which  was generated  by pushpull
> script.
> 
> Hence I tried to catalog the downloaded file and its metadata file in
> the staging area by filemgr-client and I am sure that  the file was
> ingested by filemgr:
> 
> yhkang@yunhee-kisti:~/oodt-0.5/cas-filemgr/bin$ ./filemgr-client-tes.sh
> Aug 1, 2012 7:37:37 PM
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient <init>
> INFO: Loading File Manager Configuration Properties from:
> [../etc/filemgr.properties]
> Aug 1, 2012 7:37:38 PM
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
> INFO: Running Met Extractor:
> [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for
> product type: [GenericFile]
> Aug 1, 2012 7:37:38 PM
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
> INFO: Running Met Extractor:
> [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor]
> for product type: [GenericFile]
> Aug 1, 2012 7:37:38 PM
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
> INFO: Running Met Extractor:
> [org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocationExtractor]
> for product type: [GenericFile]
> Aug 1, 2012 7:37:38 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog toDoc
> WARNING: No Metadata specified for product [tesProduct] for required
> field [DataVersion]: Attempting to continue processing metadata
> Aug 1, 2012 7:37:38 PM
> org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferer moveFile
> INFO: LocalDataTransfer: Moving File:
> file:///home/yhkang/oodt-0.5/cas-filemgr/bin/TES-Aura_L2-CO2-Nadir_r0000002147_F06_09.he5
> to file:/home/yhkang/files/tesProduct/TES-Aura_L2-CO2-Nadir_r0000002147_F06_09.he5
> Aug 1, 2012 7:37:38 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog toDoc
> WARNING: No Metadata specified for product [tesProduct] for required
> field [DataVersion]: Attempting to continue processing metadata
> ingestProduct: Result: e9f22900-dbc4-11e1-a524-6bffd22dafd0
> 
> I also checked up  the file indigested by query_tool.
> ./query_tool --url http://localhost:8000 --lucene -query
> CAS.ProductName:tesProduct
> e9f22900-dbc4-11e1-a524-6bffd22dafd0
> 
> I am wondering why the "catalog exception" was occurred as pushpull
> was running even if my file manager was working well .
> Could you recommend me any material to understand mechanism between
> filemgr and pushpull.

Hi YunHee,

This is probably a chicken and egg issue -- if your file manager catalog hasn't
been built yet and pushpull checks product uniquess the catalog won't exist
and you'll get the CatalogException the first time it's called. All times after that,
the catalog will exist and you won't get it.

Make sense?

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


Mime
View raw message