oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From YunHee Kang <yunh.k...@gmail.com>
Subject Re: Problem when running pushpull
Date Wed, 01 Aug 2012 23:20:58 GMT
Hi Chris,

I fullly made sense what you mentioned.
The root of this problem  occurred by this "Catalog exception" is
caused by insufficient configuration of my filemgr.
To be honest, my file manager has not been built yet.
But it is difficult for me to configure properties of my file manager
depending on oodt homepage.
I think what I need is some materials including examples of
configuration and test codes for them.
Let me know how to seamlessly connect two components, a file manager
and a pushpull.

Thanks,
Yunhee

2012/8/2 Mattmann, Chris A (388J) <chris.a.mattmann@jpl.nasa.gov>:
> Hey YunHee,
>
> On Aug 1, 2012, at 4:17 AM, YunHee Kang wrote:
>
>> Hi Brian,
>>
>>
>> When I was reading a file named "push_pull_framework.properties", I
>> understood  how  a metadata file is named.
>> #extension for temporary metadata file
>> org.apache.oodt.cas.pushpull.write.met.file=true
>> org.apache.oodt.cas.pushpull.met.file.extension=info.tmp
>>
>> And I found a metadata file written in XML  having its name ended with
>> ".info.tmp"  in the staging area, which  was generated  by pushpull
>> script.
>>
>> Hence I tried to catalog the downloaded file and its metadata file in
>> the staging area by filemgr-client and I am sure that  the file was
>> ingested by filemgr:
>>
>> yhkang@yunhee-kisti:~/oodt-0.5/cas-filemgr/bin$ ./filemgr-client-tes.sh
>> Aug 1, 2012 7:37:37 PM
>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient <init>
>> INFO: Loading File Manager Configuration Properties from:
>> [../etc/filemgr.properties]
>> Aug 1, 2012 7:37:38 PM
>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>> INFO: Running Met Extractor:
>> [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for
>> product type: [GenericFile]
>> Aug 1, 2012 7:37:38 PM
>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>> INFO: Running Met Extractor:
>> [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor]
>> for product type: [GenericFile]
>> Aug 1, 2012 7:37:38 PM
>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>> INFO: Running Met Extractor:
>> [org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocationExtractor]
>> for product type: [GenericFile]
>> Aug 1, 2012 7:37:38 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog toDoc
>> WARNING: No Metadata specified for product [tesProduct] for required
>> field [DataVersion]: Attempting to continue processing metadata
>> Aug 1, 2012 7:37:38 PM
>> org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferer moveFile
>> INFO: LocalDataTransfer: Moving File:
>> file:///home/yhkang/oodt-0.5/cas-filemgr/bin/TES-Aura_L2-CO2-Nadir_r0000002147_F06_09.he5
>> to file:/home/yhkang/files/tesProduct/TES-Aura_L2-CO2-Nadir_r0000002147_F06_09.he5
>> Aug 1, 2012 7:37:38 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog toDoc
>> WARNING: No Metadata specified for product [tesProduct] for required
>> field [DataVersion]: Attempting to continue processing metadata
>> ingestProduct: Result: e9f22900-dbc4-11e1-a524-6bffd22dafd0
>>
>> I also checked up  the file indigested by query_tool.
>> ./query_tool --url http://localhost:8000 --lucene -query
>> CAS.ProductName:tesProduct
>> e9f22900-dbc4-11e1-a524-6bffd22dafd0
>>
>> I am wondering why the "catalog exception" was occurred as pushpull
>> was running even if my file manager was working well .
>> Could you recommend me any material to understand mechanism between
>> filemgr and pushpull.
>
> Hi YunHee,
>
> This is probably a chicken and egg issue -- if your file manager catalog hasn't
> been built yet and pushpull checks product uniquess the catalog won't exist
> and you'll get the CatalogException the first time it's called. All times after that,
> the catalog will exist and you won't get it.
>
> Make sense?
>
> Cheers,
> Chris
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW:   http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>

Mime
View raw message