oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kang YunHee <yunh.k...@gmail.com>
Subject Re: Question about metadata specification for Filemgr and Pushpull
Date Thu, 02 Aug 2012 17:08:00 GMT
Hi Brain,

I  applied the patch(OODT-481.2012-08-01.txt) to the source of pushpull
framework, rebuilt it and swapped its jar out in my deployment according
to the following steps:

patch -p0 < OODT-481.2012-08-01.txt
mvn clean
mvn install
cd target
cp cas-pushpull-0.5-SNAPSHOT.jar ~/oodt-0.5/cas-pushpull/lib/




After I ran the pushpull script, I found out there is no "Catalog
exception" as you can see:
Aug 3, 2012 1:31:25 AM
org.apache.oodt.cas.pushpull.retrievalsystem.FileRetrievalSystem
addToDownloadQueue
WARNING: Skipping file {parent = 'null', path =
'/TES/TL2CO2N.005/2004.09.20/TES-Aura_L2-CO2-Nadir_r0000002147_F06_09.he5',
 isDir = 'false'} because it is already in staging area
PageSize: 8 PageLoc: 2
FileList size: 2
PageSize: 8 PageLoc: 952
FileList size: 952
Aug 3, 2012 1:35:38 AM
org.apache.oodt.cas.pushpull.protocol.ProtocolHandler disconnect
INFO: Disconnecting protocol
org.apache.oodt.cas.protocol.ftp.CommonsNetFtpProtocol
Aug 3, 2012 1:35:38 AM org.apache.oodt.cas.pushpull.daemon.Daemon$1 run
INFO: Daemon with ID = 90121 on RMI registry port 9012 is shutting down


But I am not sure that the file downloaded is ingested by my filemgr.
I think if it was ingested by my filemgr, it would be in the repository
directory.
However I did not find it in my repository directory:
 
Let me know how to check the status of cataloging  mentioned above.
 
Thanks,
Yunhee

On 8/2/12 6:29 AM, "Brian Foster" <holenoter@me.com> wrote:

>
>hey YunHee,
>
>I've submitted the patch, so you can instead of having to patch the code
>you can just resync your pushpull code and rebuild it and swap the jar
>out in your deployment
>
>-brian
>
>On Aug 1, 2012, at 8:28 AM, YunHee Kang wrote:
>
>> Hi Chris and Brian,
>> 
>> I am reading source codes for handling "Catalog exception" related the
>> runtime error described below.
>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException:
>> Failure writing request
>>    at 
>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.hasProduct(Xml
>>RpcFileManagerClient.java:606)
>>    at 
>>org.apache.oodt.cas.filemgr.ingest.StdIngester.hasProduct(StdIngester.jav
>>a:284)
>>    at 
>>org.apache.oodt.cas.pushpull.retrievalsystem.FileRetrievalSystem.isAlread
>>yInDatabase(FileRetrievalSystem.java:254)
>>    at 
>>org.apache.oodt.cas.pushpull.retrievalsystem.FileRetrievalSystem.addToDow
>>nloadQueue(FileRetrievalSystem.java:463)
>>    at 
>>org.apache.oodt.cas.pushpull.retrievalmethod.RemoteCrawler.processPropFil
>>e(RemoteCrawler.java:138)
>>    at 
>>org.apache.oodt.cas.pushpull.retrievalsystem.RetrievalSetup.retrieveFiles
>>(RetrievalSetup.java:109)
>>    at org.apache.oodt.cas.pushpull.daemon.Daemon$1.run(Daemon.java:218)
>>    at java.lang.Thread.run(Thread.java:662)
>> 
>> I think that the exception was caused by the wrong value of the
>> parameter productName of the method  hasProduct() in the following
>> codelet of  XmlRpcFileManagerClient.java:
>> at 
>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.hasProduct(Xml
>>RpcFileManagerClient.java:606)
>> public boolean hasProduct(String productName) throws CatalogException {
>>       Vector<Object> argList = new Vector<Object>();
>>       argList.add(productName);
>> 
>>       boolean hasProduct = false;
>> 
>>       try {
>>           hasProduct = ((Boolean) client.execute("filemgr.hasProduct",
>>                   argList)).booleanValue();
>>       } catch (XmlRpcException e) {
>>           throw new CatalogException(e.getMessage());
>>       } catch (IOException e) {
>>           throw new CatalogException(e.getMessage());
>>       }
>>       return hasProduct;
>>   }
>> 
>> I inspected that  the element  "mine-type" described below was added
>> in the file mimestypes.xml  that is  one of pushpull property files.
>> <mime-type type="product/tes">
>>           <_comment>ProductType=MyTesProductType</_comment>
>>          <glob
>> pattern="TES-Aura_L2-CO2-Nadir_r\d{10}\w{2}\d{2}\w\d{2}\.he5"
>> isregex="true"/>
>>        </mime-type>
>> 
>> I would like to know what the element "mine-type" means.
>> I am wondering how I can check the value of productName in the
>> properties of Filemgr and Pushpull.
>> 
>> I am sorry to bother you again.
>> 
>> Thanks,
>> Yunhee
>



Mime
View raw message