oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From YunHee Kang <yunh.k...@gmail.com>
Subject Re: Problem when running pushpull
Date Thu, 02 Aug 2012 02:55:55 GMT
Hi Chris,

> Have you checked out our wiki?
No, I haven't.
I just read the homepage as follows:
http://oodt.apache.org/components/maven/filemgr/

I will try to examine closely the wiki page you recommended.

Thanks,
Yunhee

2012/8/2 Mattmann, Chris A (388J) <chris.a.mattmann@jpl.nasa.gov>:
> Hi YunHee,
>
> Can you specifically tell me what's missing on the OODT file manager
> start up guide?
>
> Have you checked out our wiki?
>
> https://cwiki.apache.org/confluence/display/OODT/Home
>
> Try the documentation there, including the updated FM user guide:
>
> https://cwiki.apache.org/confluence/display/OODT/OODT+Filemgr+User+Guide
>
> Cheers,
> Chris
>
> On Aug 1, 2012, at 4:20 PM, YunHee Kang wrote:
>
>> Hi Chris,
>>
>> I fullly made sense what you mentioned.
>> The root of this problem  occurred by this "Catalog exception" is
>> caused by insufficient configuration of my filemgr.
>> To be honest, my file manager has not been built yet.
>> But it is difficult for me to configure properties of my file manager
>> depending on oodt homepage.
>> I think what I need is some materials including examples of
>> configuration and test codes for them.
>> Let me know how to seamlessly connect two components, a file manager
>> and a pushpull.
>>
>> Thanks,
>> Yunhee
>>
>> 2012/8/2 Mattmann, Chris A (388J) <chris.a.mattmann@jpl.nasa.gov>:
>>> Hey YunHee,
>>>
>>> On Aug 1, 2012, at 4:17 AM, YunHee Kang wrote:
>>>
>>>> Hi Brian,
>>>>
>>>>
>>>> When I was reading a file named "push_pull_framework.properties", I
>>>> understood  how  a metadata file is named.
>>>> #extension for temporary metadata file
>>>> org.apache.oodt.cas.pushpull.write.met.file=true
>>>> org.apache.oodt.cas.pushpull.met.file.extension=info.tmp
>>>>
>>>> And I found a metadata file written in XML  having its name ended with
>>>> ".info.tmp"  in the staging area, which  was generated  by pushpull
>>>> script.
>>>>
>>>> Hence I tried to catalog the downloaded file and its metadata file in
>>>> the staging area by filemgr-client and I am sure that  the file was
>>>> ingested by filemgr:
>>>>
>>>> yhkang@yunhee-kisti:~/oodt-0.5/cas-filemgr/bin$ ./filemgr-client-tes.sh
>>>> Aug 1, 2012 7:37:37 PM
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient <init>
>>>> INFO: Loading File Manager Configuration Properties from:
>>>> [../etc/filemgr.properties]
>>>> Aug 1, 2012 7:37:38 PM
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>>> INFO: Running Met Extractor:
>>>> [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for
>>>> product type: [GenericFile]
>>>> Aug 1, 2012 7:37:38 PM
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>>> INFO: Running Met Extractor:
>>>> [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor]
>>>> for product type: [GenericFile]
>>>> Aug 1, 2012 7:37:38 PM
>>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>>> INFO: Running Met Extractor:
>>>> [org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocationExtractor]
>>>> for product type: [GenericFile]
>>>> Aug 1, 2012 7:37:38 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog
toDoc
>>>> WARNING: No Metadata specified for product [tesProduct] for required
>>>> field [DataVersion]: Attempting to continue processing metadata
>>>> Aug 1, 2012 7:37:38 PM
>>>> org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferer moveFile
>>>> INFO: LocalDataTransfer: Moving File:
>>>> file:///home/yhkang/oodt-0.5/cas-filemgr/bin/TES-Aura_L2-CO2-Nadir_r0000002147_F06_09.he5
>>>> to file:/home/yhkang/files/tesProduct/TES-Aura_L2-CO2-Nadir_r0000002147_F06_09.he5
>>>> Aug 1, 2012 7:37:38 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog
toDoc
>>>> WARNING: No Metadata specified for product [tesProduct] for required
>>>> field [DataVersion]: Attempting to continue processing metadata
>>>> ingestProduct: Result: e9f22900-dbc4-11e1-a524-6bffd22dafd0
>>>>
>>>> I also checked up  the file indigested by query_tool.
>>>> ./query_tool --url http://localhost:8000 --lucene -query
>>>> CAS.ProductName:tesProduct
>>>> e9f22900-dbc4-11e1-a524-6bffd22dafd0
>>>>
>>>> I am wondering why the "catalog exception" was occurred as pushpull
>>>> was running even if my file manager was working well .
>>>> Could you recommend me any material to understand mechanism between
>>>> filemgr and pushpull.
>>>
>>> Hi YunHee,
>>>
>>> This is probably a chicken and egg issue -- if your file manager catalog hasn't
>>> been built yet and pushpull checks product uniquess the catalog won't exist
>>> and you'll get the CatalogException the first time it's called. All times after
that,
>>> the catalog will exist and you won't get it.
>>>
>>> Make sense?
>>>
>>> Cheers,
>>> Chris
>>>
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Chris Mattmann, Ph.D.
>>> Senior Computer Scientist
>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>> Office: 171-266B, Mailstop: 171-246
>>> Email: chris.a.mattmann@nasa.gov
>>> WWW:   http://sunset.usc.edu/~mattmann/
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> Adjunct Assistant Professor, Computer Science Department
>>> University of Southern California, Los Angeles, CA 90089 USA
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW:   http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>

Mime
View raw message