oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (3980)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: catalog queries in pge config files and filemgr-client problem
Date Mon, 31 Aug 2015 20:02:19 GMT
Val, I would inspect your file manager catalog. Can you use the OPSUI
or some other browser (Lucene) to see the metadata that is recorded
for your files? I think this is part of the problem - from the query
the only reason that $FileLocation/$Filename wouldn’t be resolved in
your queries is that they aren’t recorded as metadata fields for your

Can you check?

Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA

-----Original Message-----
From: Lewis John Mcgibbney <lewis.mcgibbney@gmail.com>
Reply-To: "dev@oodt.apache.org" <dev@oodt.apache.org>
Date: Thursday, August 27, 2015 at 5:07 PM
To: "dev@oodt.apache.org" <dev@oodt.apache.org>
Subject: Re: catalog queries in pge config files and filemgr-client problem

>Hi Val,
>Wise idea to start debugging this from the filemgr-client.
>Can you please try changing logging levels in the following file to DEBUG
>If you try your queries again then hopefully you should see some
>patterns/indications within FileMgr logs to indicate what is going on.
>On Wednesday, August 26, 2015, Mallder, Valerie
>> Hi All,
>> So, I'm phasing off of the New Horizons project for now and I am back to
>> my science data pipeline work.  I am in the process of trying to pick up
>> where I left off a few months ago - but with a twist J  I graduated from
>> RADiX and now I am trying to get my pipeline system set up to run with
>> 'regular' OODT.  I have made extremely good progress so far. I have
>> much replicated all of the functionality of filemgr and workflow manager
>> that I had been using with RADiX, with the exception of one thing that
>> doesn't seem to be working the way it used too.  And I'm hoping someone
>> there can give me some ideas on how to fix it.
>> I have a catalog query in one of my PGE config files. I copied the
>> for the query statement directly from one of Chris' PGE config files
>> (PgeConfig_RatAggregator.xml) in the 'drat-master' application.
>> Here is Chris' statement:
>>     <metadata key="InputFiles"
>> FileLocation,Filename,CAS.ProductReceivedTime FROM RatLog}"/>
>> Here is my statement:
>>      <metadata key="InitialDirList"
>> FileLocation,Filename,CAS.ProductReceivedTime FROM ScienceFile}"/>
>> And here are the results from the query:
>> And those results are not exactly what I wanted J
>> So I ran some experiments with ./filemgr-client on the command line to
>> make sure the files I expected to be in the catalog were in fact in the
>> catalog.
>> The following operation worked just as expected.  The expected number of
>> products were listed.
>> ./filemgr-client --url http://localhost:9016 --operation --getFirstPage
>> --productTypeName ScienceFile
>> However, this query gave some unexpected results. And this may be
>> the problem with the query in my PGE config file, but I am not sure.
>> ./filemgr-client --url http://localhost:9016 --operation -sqlQuery
>> "SELECT FileLocation,Filename FROM ScienceFile"
>> That query only outputs several blank lines to the screen and nothing
>> I am not very familiar with all of the catalog related code yet so I am
>> not sure where to start looking for the problem.  If you know this area
>> the code well and have some debugging ideas, please let me know.
>> Thanks so much!
>> Val

View raw message