oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (388J)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: Issue with File Manager initiating a Lucene Catalog
Date Tue, 21 Feb 2012 21:50:56 GMT
Hi Luca,

+1 to filing an issue to fix this in 0.5...

Cheers,
Chris

On Feb 21, 2012, at 7:26 AM, Cinquini, Luca (3880) wrote:

> Hi Sean and Chris,
>        on the same topic, I think the desired behavior would be that if the FM is configured
to use Lucene in the back-end, an empty but usable catalog would be created at startup, without
having to wait for the first ingestion. Otherwise, as Sean noted, you can't query the FM until
something gets ingested.
> If nobody disagree, I can file a Jira issue for 0.5 ?
> thanks, Luca
> 
> On Feb 21, 2012, at 12:24 AM, Mattmann, Chris A (388J) wrote:
> 
>> Hey Sean,
>> 
>> Looking at the stack trace below, it looks like you had an existing catalog directory,
but it didn't have anything inside of it.
>> 
>> I think you need to *first* delete the catalog directory, restart the file manager,
and then try again. It should work at that point,
>> and that was the same behavior from 0.3.
>> 
>> Let me know if that addresses it.
>> 
>> Cheers,
>> Chris
>> 
>> On Feb 20, 2012, at 6:57 PM, Hardman, Sean H (388J) wrote:
>> 
>>> Sorry for spamming the list today but I have been doing some comparative tests
today with the File Manager. When populating a newly installed version (0.4-SNAPSHOT) of the
File Manager, I was unable to ingest a product until I supplied an existing (yet empty of
products) Lucene Catalog for the File Manager to populate. I have included the stack trace
from both the client and the server below. This isn't the behavior seen in the 0.3 release.
If the Lucene Catalog does not exist in 0.3, it is created upon the first file ingestion.
>>> 
>>> Once again, I am not sure whether this is worthy of a JIRA issue. I am just pointing
out differing behavior.
>>> 
>>> Thanks,
>>> Sean
>>> 
>>> 
>>> Stack trace from filemgr-client:
>>> 
>>> bash-3.2$ ./filemgr-client --url http://localhost:9000 --operation --ingestProduct
--productName Test --productStructure Flat --productTypeName CatalogFile --metadataFile "file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt.met"
--refs "file:///Users/shardman/deploy/pds-build2b/storage-0.2.0/test/test.txt"
>>> Feb 20, 2012 6:49:30 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient
<init>
>>> INFO: Loading File Manager Configuration Properties from: [../etc/filemgr.properties]
>>> org.apache.xmlrpc.XmlRpcException: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException:
Error ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to
index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file
or directory)
>>> at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
>>> at org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
>>> at org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.ingestProduct(XmlRpcFileManagerClient.java:1162)
>>> at org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction.execute(IngestProductCliAction.java:102)
>>> at org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:296)
>>> at org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:179)
>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.main(XmlRpcFileManagerClient.java:1307)
>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient
ingestProduct
>>> SEVERE: Failed to ingest product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4]
: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error
ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index
product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
-- rolling back ingest
>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient
ingestProduct
>>> SEVERE: Failed to rollback ingest of product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4]
: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error
ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index
product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> ERROR: Failed to ingest product 'Test' : Failed to ingest product [org.apache.oodt.cas.filemgr.structs.Product@535c97e4]
: java.lang.Exception: org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error
ingesting product [org.apache.oodt.cas.filemgr.structs.Product@3e4a9a7d] : Unable to index
product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> 
>>> Stack trace from filemgr:
>>> 
>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager
runExtractors
>>> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor]
for product type: [CatalogFile]
>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager
runExtractors
>>> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor]
for product type: [CatalogFile]
>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager
runExtractors
>>> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocationExtractor]
for product type: [CatalogFile]
>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog addCompleteProductToIndex
>>> WARNING: Unable to index product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments
(No such file or directory)
>>> java.io.FileNotFoundException: /Users/shardman/deploy/oodt/catalog/segments (No
such file or directory)
>>> at java.io.RandomAccessFile.open(Native Method)
>>> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216)
>>> at org.apache.lucene.store.FSIndexInput$Descriptor.<init>(FSDirectory.java:430)
>>> at org.apache.lucene.store.FSIndexInput.<init>(FSDirectory.java:439)
>>> at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:329)
>>> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:45)
>>> at org.apache.lucene.index.IndexWriter$1.doBody(IndexWriter.java:264)
>>> at org.apache.lucene.store.Lock$With.run(Lock.java:99)
>>> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:259)
>>> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:204)
>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addCompleteProductToIndex(LuceneCatalog.java:990)
>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addProductReferences(LuceneCatalog.java:336)
>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.addProductReferences(XmlRpcFileManager.java:1097)
>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRpcFileManager.java:740)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>>> at java.lang.Thread.run(Thread.java:680)
>>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Unable to index
product: [Test]: Message: /Users/shardman/deploy/oodt/catalog/segments (No such file or directory)
>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addCompleteProductToIndex(LuceneCatalog.java:1005)
>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.addProductReferences(LuceneCatalog.java:336)
>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.addProductReferences(XmlRpcFileManager.java:1097)
>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRpcFileManager.java:740)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>>> at java.lang.Thread.run(Thread.java:680)
>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog removeProductDocument
>>> WARNING: Exception removing product: [Test] from index: Message: /Users/shardman/deploy/oodt/catalog/segments
(No such file or directory)
>>> Feb 20, 2012 6:49:31 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager
removeProduct
>>> WARNING: Exception modifying product: [null]: Message: /Users/shardman/deploy/oodt/catalog/segments
(No such file or directory)
>>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: /Users/shardman/deploy/oodt/catalog/segments
(No such file or directory)
>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.removeProductDocument(LuceneCatalog.java:962)
>>> at org.apache.oodt.cas.filemgr.catalog.LuceneCatalog.removeProduct(LuceneCatalog.java:298)
>>> at org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.removeProduct(XmlRpcFileManager.java:926)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
>>> at org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
>>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
>>> at org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
>>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
>>> at java.lang.Thread.run(Thread.java:680)
>>> 
>> 
>> 
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW:   http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> 
> 


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


Mime
View raw message