Mailing-List: contact dev-help@oodt.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@oodt.apache.org
Received-SPF: pass (nike.apache.org: local policy)
From: "Mattmann, Chris A (388J)" <chris.a.mattmann@jpl.nasa.gov>
To: "<dev@oodt.apache.org>" <dev@oodt.apache.org>
Subject: Re: Problem when running pushpull
Thread-Topic: Problem when running pushpull
Thread-Index: AQHNb9crbC3sNbVvx02VRcciLxoZsJdFtCcAgABaIACAADj5gA==
Date: Thu, 2 Aug 2012 02:42:53 +0000
Message-ID: <3FCBE2F4-47DD-4360-96AA-A4ECA424C08A@jpl.nasa.gov>
References: 
 <CAH=8jEAgAZHv_bt0SgUoZ2WB24SXzJ9w-O8Bc5wWPZ5Su1iuCw@mail.gmail.com>
 <a3cfc53f-50d8-63e4-e0ca-b12b44b18d0d@me.com>
 <CAH=8jECemd1dMmzgndBEVmkjaUe9GDTxJt8HXpSjAJfmkjcwow@mail.gmail.com>
 <CAH=8jEBBb0FVkjz_xW2+gbGSD9eRzKkNZCOqmGEHzVoGAd5fSw@mail.gmail.com>
 <CAH=8jEDzJVd_pO=8SuA8Ytt6g5oJ-5bhEhX2xqzMYZ1rzyzeug@mail.gmail.com>
 <2909B3C8-3262-4E7B-9718-BD22CABF91C0@jpl.nasa.gov>
 <CAH=8jECvstVfWAKt5R8MNw12smf7GzZFCBVKbXnrib-rqWSWwQ@mail.gmail.com>
In-Reply-To: 
 <CAH=8jECvstVfWAKt5R8MNw12smf7GzZFCBVKbXnrib-rqWSWwQ@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <106511BFF1CC1E4491119FA39589859C@ad.jpl>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Hi YunHee,

Can you specifically tell me what's missing on the OODT file manager
start up guide?

Have you checked out our wiki?

https://cwiki.apache.org/confluence/display/OODT/Home

Try the documentation there, including the updated FM user guide:

https://cwiki.apache.org/confluence/display/OODT/OODT+Filemgr+User+Guide

Cheers,
Chris

On Aug 1, 2012, at 4:20 PM, YunHee Kang wrote:

> Hi Chris,
>=20
> I fullly made sense what you mentioned.
> The root of this problem  occurred by this "Catalog exception" is
> caused by insufficient configuration of my filemgr.
> To be honest, my file manager has not been built yet.
> But it is difficult for me to configure properties of my file manager
> depending on oodt homepage.
> I think what I need is some materials including examples of
> configuration and test codes for them.
> Let me know how to seamlessly connect two components, a file manager
> and a pushpull.
>=20
> Thanks,
> Yunhee
>=20
> 2012/8/2 Mattmann, Chris A (388J) <chris.a.mattmann@jpl.nasa.gov>:
>> Hey YunHee,
>>=20
>> On Aug 1, 2012, at 4:17 AM, YunHee Kang wrote:
>>=20
>>> Hi Brian,
>>>=20
>>>=20
>>> When I was reading a file named "push_pull_framework.properties", I
>>> understood  how  a metadata file is named.
>>> #extension for temporary metadata file
>>> org.apache.oodt.cas.pushpull.write.met.file=3Dtrue
>>> org.apache.oodt.cas.pushpull.met.file.extension=3Dinfo.tmp
>>>=20
>>> And I found a metadata file written in XML  having its name ended with
>>> ".info.tmp"  in the staging area, which  was generated  by pushpull
>>> script.
>>>=20
>>> Hence I tried to catalog the downloaded file and its metadata file in
>>> the staging area by filemgr-client and I am sure that  the file was
>>> ingested by filemgr:
>>>=20
>>> yhkang@yunhee-kisti:~/oodt-0.5/cas-filemgr/bin$ ./filemgr-client-tes.sh
>>> Aug 1, 2012 7:37:37 PM
>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient <init>
>>> INFO: Loading File Manager Configuration Properties from:
>>> [../etc/filemgr.properties]
>>> Aug 1, 2012 7:37:38 PM
>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>> INFO: Running Met Extractor:
>>> [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for
>>> product type: [GenericFile]
>>> Aug 1, 2012 7:37:38 PM
>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>> INFO: Running Met Extractor:
>>> [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtra=
ctor]
>>> for product type: [GenericFile]
>>> Aug 1, 2012 7:37:38 PM
>>> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>>> INFO: Running Met Extractor:
>>> [org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLoca=
tionExtractor]
>>> for product type: [GenericFile]
>>> Aug 1, 2012 7:37:38 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalo=
g toDoc
>>> WARNING: No Metadata specified for product [tesProduct] for required
>>> field [DataVersion]: Attempting to continue processing metadata
>>> Aug 1, 2012 7:37:38 PM
>>> org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferer moveFile
>>> INFO: LocalDataTransfer: Moving File:
>>> file:///home/yhkang/oodt-0.5/cas-filemgr/bin/TES-Aura_L2-CO2-Nadir_r000=
0002147_F06_09.he5
>>> to file:/home/yhkang/files/tesProduct/TES-Aura_L2-CO2-Nadir_r0000002147=
_F06_09.he5
>>> Aug 1, 2012 7:37:38 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalo=
g toDoc
>>> WARNING: No Metadata specified for product [tesProduct] for required
>>> field [DataVersion]: Attempting to continue processing metadata
>>> ingestProduct: Result: e9f22900-dbc4-11e1-a524-6bffd22dafd0
>>>=20
>>> I also checked up  the file indigested by query_tool.
>>> ./query_tool --url http://localhost:8000 --lucene -query
>>> CAS.ProductName:tesProduct
>>> e9f22900-dbc4-11e1-a524-6bffd22dafd0
>>>=20
>>> I am wondering why the "catalog exception" was occurred as pushpull
>>> was running even if my file manager was working well .
>>> Could you recommend me any material to understand mechanism between
>>> filemgr and pushpull.
>>=20
>> Hi YunHee,
>>=20
>> This is probably a chicken and egg issue -- if your file manager catalog=
 hasn't
>> been built yet and pushpull checks product uniquess the catalog won't ex=
ist
>> and you'll get the CatalogException the first time it's called. All time=
s after that,
>> the catalog will exist and you won't get it.
>>=20
>> Make sense?
>>=20
>> Cheers,
>> Chris
>>=20
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW:   http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>=20


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++