oodt-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stough, Timothy M (388F)" <timothy.m.sto...@jpl.nasa.gov>
Subject Re: Two File Manager Questions
Date Mon, 21 Nov 2011 17:25:19 GMT
Cameron,

Glad to hear that this lead to a fix.  Chris and I reverse engineered the xmlrpc serialization
of query objects to get around this.

Does this bug apply to queries submitted directly as query objects?

Thanks,
Tim.

On Nov 17, 2011, at 10:34 PM, Cameron Goodale wrote:
> Mattmann and I worked out the issue regarding underscores and hyphens.  I logged a JIRA
issue here: 
> 
> https://issues.apache.org/jira/browse/OODT-350
> 
> and uploaded a patch, so you can patch your instance and get back to running queries.
 This bug fix will be added into the next release of Apache OODT.
> 
> Thanks!
> 
> 
> Cameron
> 
> P.S.  The reason Cecilia's code works is because it was patched by Brian Foster.  In
fact this patch is just a port of his code from branch back to trunk.  So thanks to B.Foster
and Enjoy!
> 
> On Thu, Nov 17, 2011 at 8:18 PM, Cameron Goodale <sigep311@gmail.com> wrote:
> Tim,
> 
> I just encountered the same issue when querying for a 'ProductType:TCCON-garmisch'. 
This is throwing the same errors you got:
> 
> WARNING: Query: [q=ProductType:TCCON AND ProductType:garmisch] for Product Type: [urn:co2:TCCON-garmisch]
returned no results
> Nov 17, 2011 7:46:45 PM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog paginateQuery
> 
> As best I can figure the hyphen and underscore must be some kind of reserved character
in Lucene.  We are running an older cas-filemgr-0.3-SNAPSHOT.jar on Linux CentOS 5.7
> 
> Has anyone else encountered this issue besides Tim?  Also Tim what is the OS you are
running on with your problem?
> 
> Cecilia,
> 
> What OS version are you running on where this is not an issue?  Also what version of
fileManager are you using??
> 
> Thanks!
> 
> 
> 
> Cameron
> 
> 
> 
> On Wed, Nov 9, 2011 at 10:40 AM, Stough, Timothy M (388F) <timothy.m.stough@jpl.nasa.gov>
wrote:
> Cecilia,
> 
> Thanks for the reply.  I tried your suggested query and got similar results as before.
 Here's what I tried...
> 
> In preparation, I ingested a blank text file with the same product name as in your example:
> 
> [stough@aria1 bin]$ ./filemgr-client --url http://localhost:9000 --operation --ingestProduct
--productName acos_qSnd_090627_04_Production_v050050_Sel2900_r05_111109172137i.txt --productStructure
Flat --productTypeName GenericFile --metadataFile file:///proj/aria/cas/data-system/bin/blah.txt.met
--refs file:///proj/aria/cas/data-system/bin/blah.txt
> Nov 9, 2011 10:32:35 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient <init>
> INFO: Loading File Manager Configuration Properties from: [../etc/filemgr.properties]
> ingestProduct: Result: 31f26edf-0b01-11e1-b910-4da260ad04a8
> 
> It was properly ingested as indicated in the logs:
> 
> Nov 9, 2011 10:32:35 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor]
for product type: [GenericFile]
> Nov 9, 2011 10:32:35 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor]
for product type: [GenericFile]
> Nov 9, 2011 10:32:35 AM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocationExtractor]
for product type: [GenericFile]
> Nov 9, 2011 10:32:35 AM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog toDoc
> WARNING: No Metadata specified for product [acos_qSnd_090627_04_Production_v050050_Sel2900_r05_111109172137i.txt]
for required field [DataVersion]: Attempting to continue processing metadata
> Nov 9, 2011 10:32:35 AM org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferer
moveFile
> INFO: LocalDataTransfer: Moving File: file:///proj/aria/cas/data-system/bin/blah.txt
to file:/proj/aria/repository/files/acos_qSnd_090627_04_Production_v050050_Sel2900_r05_111109172137i.txt/blah.txt
> Nov 9, 2011 10:32:35 AM org.apache.oodt.cas.filemgr.catalog.LuceneCatalog toDoc
> WARNING: No Metadata specified for product [acos_qSnd_090627_04_Production_v050050_Sel2900_r05_111109172137i.txt]
for required field [DataVersion]: Attempting to continue processing metadata
> 
> I tried the query that you suggested with a few minor mods for my install:
> 
> ./query_tool --url http://localhost:9000 --lucene -query CAS.ProductName:acos_qSnd_090627_04_Production_v050050_Sel2900_r05_111109172137i.txt
> 
> and got the following in my File Manager logs:
> 
> datacasting 3.1.0cWARNING: Query: [q=CAS.ProductName:acos AND CAS.ProductName:qSnd_090627_04
AND CAS.ProductName:Production_v050050_Sel2900_r05_111109172137i AND CAS.ProductName:txt]
for Product Type: [urn:oodt:GenericFile] returned no results
> 
> The addition of single, or double quotes had no effect.
> 
> Anymore thoughts?
> 
> Thanks,
> Tim.
> 
> On Nov 9, 2011, at 9:52 AM, Cheng, Cecilia S (388K) wrote:
> > Looks like emails to user@oodt.apache.org take longer to arrive, so I’m sending
this directly to you. Hope it helps.
> >
> > -- cecilia
> >
> > ------ Forwarded Message
> > From: "Cheng, Cecilia S (388K)" <cecilia.s.cheng@jpl.nasa.gov>
> > Date: Wed, 9 Nov 2011 09:30:37 -0800
> > To: "user@oodt.apache.org" <user@oodt.apache.org>
> > Subject: Re: Two File Manager Questions
> >
> > Hi Tim,
> >
> > > B) Disallowed characters in product names?
> > >
> > >  My product names look like this:  "ALPSRP225250610-H1.0__A"  If I try to
> > > search for a product by name using a Lucene query through query_tool, the
> > > query breaks across the "_".  What I see in the log is:
> > >
> > > WARNING: Query: [q=Filename:ALPSRP154650650-H1.0 AND Filename:A] for Product
> > > Type: [urn:oodt:GenericFile] returned no results
> > >
> > > So it looks like the __ gets turned into an "AND".  What's the deal and how
do
> > > I fix it?
> >
> > The '_' in the product name works for us. See:
> >
> >> $ ./query_tool --url $FILEMGR_URL --lucene -query FILENAME:acos_qSnd_090627_04_Production_v050050_Sel2900_r05_111109172137i.txt
> >> 7151
> >
> > Maybe you can try putting single quotes around the file name. See:
> >
> >> $ ./query_tool --url $FILEMGR_URL --lucene -query FILENAME:'acos_qSnd_090627_04_Production_v050050_Sel2900_r05_111109172137i.txt'
> >> 7151
> >
> > -- cecilia
> >
> >
> > ------ End of Forwarded Message
> 
> -----------------------------------------------------------------
> Tim Stough
> NASA/Caltech Jet Propulsion Lab
> Senior System Architect
> Data Understanding Group (Section 388)
> 818-393-5347 (office)
> 626-644-6574 (cell)
> -----------------------------------------------------------------
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> 
> Sent from a Tin Can attached to a String
> 
> 
> 
> 
> 
> -- 
> 
> Sent from a Tin Can attached to a String
> 
> 

-----------------------------------------------------------------
Tim Stough
NASA/Caltech Jet Propulsion Lab
Senior System Architect
Data Understanding Group (Section 388)
818-393-5347 (office)
626-644-6574 (cell)
-----------------------------------------------------------------






Mime
View raw message