oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mallder, Valerie" <Valerie.Mall...@jhuapl.edu>
Subject RE: catalog queries in pge config files and filemgr-client problem
Date Mon, 31 Aug 2015 21:41:13 GMT
Hi Lewis,

Thanks Lewis, I tried setting it to DEBUG, that it told me it wasn't a recognized level. But,
I used FINEST and added more log message and eventually found that I had run into the issue
that has already been documented in OODT-854 which is supposed to be fixed in 0.10.  It has
to do with where I was putting my policy files for the filemgr.  It sounds like the fix for
OODT-854 will solve my problem.  Once I put all of my policy files in FILEMGR_HOME/policy/core,
the catalog query started working. 

Thanks,
Val




> -----Original Message-----
> From: Lewis John Mcgibbney [mailto:lewis.mcgibbney@gmail.com]
> Sent: Thursday, August 27, 2015 8:07 PM
> To: dev@oodt.apache.org
> Subject: Re: catalog queries in pge config files and filemgr-client problem
> 
> Hi Val,
> Wise idea to start debugging this from the filemgr-client.
> Can you please try changing logging levels in the following file to DEBUG
> 
> https://github.com/apache/oodt/blob/trunk/filemgr/src/main/resources/logging.pro
> perties#L40-L74
> 
> If you try your queries again then hopefully you should see some
> patterns/indications within FileMgr logs to indicate what is going on.
> 
> 
> On Wednesday, August 26, 2015, Mallder, Valerie <Valerie.Mallder@jhuapl.edu>
> wrote:
> 
> > Hi All,
> >
> > So, I'm phasing off of the New Horizons project for now and I am back
> > to my science data pipeline work.  I am in the process of trying to
> > pick up where I left off a few months ago - but with a twist J  I
> > graduated from RADiX and now I am trying to get my pipeline system set
> > up to run with the 'regular' OODT.  I have made extremely good
> > progress so far. I have pretty much replicated all of the
> > functionality of filemgr and workflow manager that I had been using
> > with RADiX, with the exception of one thing that doesn't seem to be
> > working the way it used too.  And I'm hoping someone out there can give me
> some ideas on how to fix it.
> >
> > I have a catalog query in one of my PGE config files. I copied the
> > syntax for the query statement directly from one of Chris' PGE config
> > files
> > (PgeConfig_RatAggregator.xml) in the 'drat-master' application.
> >
> > Here is Chris' statement:
> >
> >     <metadata key="InputFiles"
> > val="SQL(FORMAT='$FileLocation/$Filename',SORT_BY='CAS.ProductReceived
> > Time'){SELECT FileLocation,Filename,CAS.ProductReceivedTime FROM
> > RatLog}"/>
> >
> > Here is my statement:
> >
> >      <metadata key="InitialDirList"
> > val="SQL(FORMAT='$FileLocation/$Filename',SORT_BY='CAS.ProductReceived
> > Time'){SELECT FileLocation,Filename,CAS.ProductReceivedTime FROM
> > ScienceFile}"/>
> >
> > And here are the results from the query:
> >
> >
> > $FileLocation/$Filename,$FileLocation/$Filename,$FileLocation/$Filenam
> > e,$FileLocation/$Filename,$FileLocation/$Filename,$FileLocation/$Filen
> > ame,$FileLocation/$Filename,$FileLocation/$Filename,$FileLocation/$Fil
> > ename,$FileLocation/$Filename,$FileLocation/$Filename,$FileLocation/$F
> > ilename,$FileLocation/$Filename,$FileLocation/$Filename,$FileLocation/
> > $Filename,$FileLocation/$Filename,$FileLocation/$Filename,$FileLocatio
> > n/$Filename,$FileLocation/$Filename,$FileLocation/$Filename,$FileLocat
> > ion/$Filename,$FileLocation/$Filename,$FileLocation/$Filename,$FileLoc
> > ation/$Filename,$FileLocation/$Filename,$FileLocation/$Filename,$FileL
> > ocation/$Filename,$FileLocation/$Filename,
> >
> > And those results are not exactly what I wanted J
> >
> > So I ran some experiments with ./filemgr-client on the command line to
> > make sure the files I expected to be in the catalog were in fact in
> > the catalog.
> >
> > The following operation worked just as expected.  The expected number
> > of products were listed.
> > ./filemgr-client --url http://localhost:9016 --operation
> > --getFirstPage --productTypeName ScienceFile
> >
> > However, this query gave some unexpected results. And this may be
> > related the problem with the query in my PGE config file, but I am not sure.
> > ./filemgr-client --url http://localhost:9016 --operation -sqlQuery
> > -query "SELECT FileLocation,Filename FROM ScienceFile"
> >
> > That query only outputs several blank lines to the screen and nothing else.
> >
> > I am not very familiar with all of the catalog related code yet so I
> > am not sure where to start looking for the problem.  If you know this
> > area of the code well and have some debugging ideas, please let me know.
> >
> > Thanks so much!
> >
> > Val
> >
> >
Mime
View raw message