oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mallder, Valerie" <Valerie.Mall...@jhuapl.edu>
Subject RE: more info for OODT-751 OPSUI Pages constantly expire
Date Thu, 13 Nov 2014 17:20:53 GMT
Hi Tyler,

Thanks for writing!  I have heard of DRAT (Chris pointed me to it for a examples of a running
PGE) but I have not used it and I don't really know what it does.  If you think it could help
me figure out something, and there is a good source of documentation to help me use it, I
will be happy to try it.

Do you happen to know what "GANGLIA" is and what default "url" I should set the GANGLIA_URL
environment too?  I see this variable in the http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/webapp/META-INF/context.xml
file and this is the first I have seen of it.

I've been searching the mailing list archives for any information I can find on opsui, and
I found out that there are several ways to browse for products and files. All of which are
greek to me. So, I have no idea where to start in trying to use any one of them. While I am
familiar with html, css, php, mysql, and a little bit of javascript for building websites,
I have never used tomcat and I don't know what a "web app", a "wicket web app" or a "REST
service" are. (maybe I do, but by a different name?) So, I'm a bit at a disadvantage here.

Thanks,
Val




Valerie A. Mallder
New Horizons Deputy Mission System Engineer
Johns Hopkins University/Applied Physics Laboratory


> -----Original Message-----
> From: Tyler Palsulich [mailto:tpalsulich@gmail.com]
> Sent: Thursday, November 13, 2014 11:36 AM
> To: dev
> Subject: RE: more info for OODT-751 OPSUI Pages constantly expire
> 
> Have you tried installing and running DRAT?
> 
> Tyler
> On Nov 13, 2014 8:11 AM, "Mallder, Valerie" <Valerie.Mallder@jhuapl.edu>
> wrote:
> 
> > For those of you who are using the pcs-opsui in an operational
> > environment without getting expired pages please tell me what version
> > you are using, the OS you are running it on, and the browser you are using.
> >
> > I need to demonstrate "something" to my project manager to show the
> > usefulness of OODT in the Jedi instrument science data pipeline here
> > at APL. And right now, I have nothing to "show" for my last few months
> > of work. If someone can tell me a configuration using opsui that is
> > working then maybe I can try to mimic that and get something useful up and
> running.
> > So far, for me, versions 0.6, 0.7 and the current trunk are showing
> > only expired pages.
> >
> > Thanks very much!
> > Val
> >
> >
> >
> > Sent from my iPhone.
> > ________________________________
> > From: Mallder, Valerie <Valerie.Mallder@jhuapl.edu>
> > Sent: Tuesday, November 11, 2014 2:55:12 PM
> > To: dev@oodt.apache.org
> > Subject: more info for OODT-751 OPSUI Pages constantly expire
> >
> > Hi Chris,
> >
> > I know you are working on this, but I wanted to let you know that I
> > tried to use version 0.6 and am having the same problem in version
> > 0.6. But, I can't guarantee that I installed version 0.6 correctly so
> > I would like to run this by you.  Based on your email below here's what I did:
> >
> > 1. Downloaded pcs-opsui-0.76.war from:
> > http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/
> > and saved it in a new folder named $OODT_HOME/bin/opsui
> >
> > 2. Created a script called $OODT_HOME/bin/opsui/runopsui that sets the
> > following variables (this is output from my script):
> > Using OODT_BASE:
> >  /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy
> > Using OODT_HOME:
> >  /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy
> > Using OODT_TMPDIR:
> >  /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/temp
> > Using FILEMGR_URL:       http://localhost:9000
> > Using WORKFLOW_URL:      http://localhost:9001
> > Using RESMGR_URL:        http://localhost:9002
> > Using WORKFLOW_HOME:
> >  /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/workflow
> > Using RESMGR_HOME:
> >  /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/resmgr
> > Using CRAWLER_HOME:
> > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/crawler
> > Using TOMCAT_HOME:
> >  /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/tomcat
> > Using PCS_HOME:
> > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pcs
> > Using PGE_HOME:
> > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pge
> > Using PGE_JOBS_DIR:
> > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/
> > jobs
> > Using FEI_DROP_DIR:
> > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/tele
> > metry
> > Using JEDI_L0_DIR:
> >  /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l0
> > Using JEDI_L2_DIR:
> >  /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l2
> > Using ARCHIVE_DIR:
> >
> > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/arch
> > ive
> > Using BACKUP_DIR:
> > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/met
> > Using FAILURE_DIR:
> >
> > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/fail
> > ure Using JEDI_PIPELINE_DIR: /homes/malldva1/working/pipeline Using
> > SNAPSHOT_DIR:
> > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/
> > jobs/snapshot
> >
> > (Then, I followed the steps at the bottom of:
> > https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+O
> > PSUI
> > )
> >
> > 3. Downloaded
> > http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/webapp/M
> > ETA-INF/context.xml And saved it as $OODT_HOME/bin/opsui/pcs-opsui.xml
> >
> > 4. Edited $OODT_HOME/bin/opsui/pcs-opsui.xml and changed the first
> > line
> > from:
> > <Context path="/pcs-opsui">
> > To
> > <Context path="/pcs-opsui"
> > docBase="[OODT_HOME]/bin/opsui/pcs-opsui-0.6.war">
> >
> > 5.  Killed any process with "tomcat" in its name to ensure that tomcat
> > is not running.
> >
> > 6. Executed the following command to create a symbolic link:
> > cd $OODT_HOME/bin/opsui/
> > ln -s pcs-opsui.xml to
> > $TOMCAT_HOME/conf/Catalina/localhost/pcs-opsui.xml
> >
> > 7. Then, in my script that sets the environment variables, I added the
> > following command to start tomcat.
> > exec "$OODT_BASE"/tomcat/bin/catalina.sh start
> >
> > 8. Then, I ran my new $OODT_HOME/bin/opsui/runopsui script
> >
> > 9. Then, I started firefox  and went to "localhost:8080/pcs-opsui"
> >
> >
> >
> > The home page comes up, and the PCS Status page comes up, but all
> > other pages are expired.  I should note that the "radix" installation
> > of version
> > 0.7 points me to localhost:8080/opsui, while step 10 of the
> > instructions from the quick start guide of the wiki point me to
> > localhost:8080/pcs-opsui. And, no matter which one I go to, the pages
> > expire.
> >
> > I also tried clearing out my cache and browser history and restarting
> > firefox and topcat, etc. and nothing seems to help.
> >
> > If I have gotten the version 0.6 up and running correctly, I thought
> > you would be interested to know that I am seeing the error. If I did
> > not get version 0.6 up and running, then I need more hints on how to
> > do it.  :)
> >
> > Thanks,
> > Val
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Valerie A. Mallder
> > New Horizons Deputy Mission System Engineer Johns Hopkins
> > University/Applied Physics Laboratory
> >
> >
> > > -----Original Message-----
> > > From: Mattmann, Chris A (3980)
> > > [mailto:chris.a.mattmann@jpl.nasa.gov]
> > > Sent: Friday, October 03, 2014 1:07 PM
> > > To: dev@oodt.apache.org
> > > Subject: Re: Success! RE: how to use MetadataBasedFileVersioner
> > > properly
> > >
> > > Awesome Val! :)
> > >
> > > I think you¹re running into this:
> > >
> > > https://issues.apache.org/jira/browse/OODT-751
> > >
> > >
> > > In the meanwhile, try the 0.6 OPSUI, which you can grab from here:
> > >
> > > http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/
> > >
> > >
> > > Grab the WAR file and drop it into your favorite container.
> > > Make sure you have all of these environment variables installed
> > > *before* starting Tomcat or Jetty, etc.:
> > >
> > >
> > https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+O
> > PSUI
> > >
> > >
> > > (see steps at bottom and replace 0.5 with 0.6)
> > >
> > > I¹m working on a fix for OODT-751, at which point RADIX will be
> > > pretty
> > buff. Next
> > > steps at that point:
> > >
> > > 1. Release 0.7 and then encourage folks to get started by using the
> > Vagrant build,
> > > e.g.,
> > >
> > > git clone https://github.com/apache/oodt cd vagrant/radix vagrant up
> > >
> > > 2. Fix OODT-491 and remaining workflow manager issues for Wengine 3.
> > > Work on Streaming OODT API with AMP Stack (via M. Starch et al) and
> > > release
> > in 0.8.
> > > 4. Conquer and win.
> > >
> > > Thanks!
> > >
> > > Cheers,
> > > Chris
> > >
> > >
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > +++++
> > > Chris Mattmann, Ph.D.
> > > Chief Architect
> > > Instrument Software and Science Data Systems Section (398) NASA Jet
> > > Propulsion Laboratory Pasadena, CA 91109 USA
> > > Office: 168-519, Mailstop: 168-527
> > > Email: chris.a.mattmann@nasa.gov
> > > WWW:  http://sunset.usc.edu/~mattmann/
> > >
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > +++++
> > > Adjunct Associate Professor, Computer Science Department University
> > > of Southern California, Los Angeles, CA 90089 USA
> > >
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > +++++
> > >
> > >
> > >
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: <Mallder>, Valerie <Valerie.Mallder@jhuapl.edu>
> > > Reply-To: "dev@oodt.apache.org" <dev@oodt.apache.org>
> > > Date: Friday, October 3, 2014 at 9:53 AM
> > > To: "dev@oodt.apache.org" <dev@oodt.apache.org>
> > > Subject: Success!  RE: how to use MetadataBasedFileVersioner
> > > properly
> > >
> > > >Hi Chris,
> > > >
> > > >Yes, that indeed fixed it!  Thanks so much!  I now have 18
> > > >engineering files ingested. Whoo hoo!!
> > > >
> > > >Ok, so now, I would like to see what the opsui has to say about my
> > > >ingested files. And here comes probably a really stupid question.
> > > >I am using Firefox on a Redhat Linux box, and I'm not that familiar
> > > >with Firefox and it's settings, so this might be a browser setting issue.
> > > >When I startup the opsui and select "File Catalog Browse" it shows
> > > >that I have
> > > >18 EngineeringFiles. Then, when I select File Catalog Browse from
> > > >the strip of options under the logo, I get a message saying the
> > > >page has expired. And here's the link that shows up in the address bar:
> > > >http://localhost:8080/opsui/?wicket:interface=:7:fmbrowser_link::IL
> > > >inkL
> > > >ist
> > > >ener::
> > > >
> > > >And I get the page expired message for all of the options that I select.
> > > >Any idea's on this one??
> > > >
> > > >In the meantime though, I will start playing with an action for
> > > >post ingestion success that simply makes another copy of all these
> > > >engineering files and puts them another folder for Level 0 files.
> > > >
> > > >Thanks,
> > > >Val
> > > >
> > > >
> > > >
> > > >
> > > >Valerie A. Mallder
> > > >New Horizons Deputy Mission System Engineer Johns Hopkins
> > > >University/Applied Physics Laboratory
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: Chris Mattmann [mailto:chris.mattmann@gmail.com]
> > > >> Sent: Friday, October 03, 2014 3:03 AM
> > > >> To: dev@oodt.apache.org
> > > >> Subject: Re: how to use MetadataBasedFileVersioner properly
> > > >>
> > > >> You?re almost there Val!
> > > >>
> > > >> Unfortunately Versioners right now aren?t configurable from
> > > >>product type policy  (would be great to capture this in a JIRA issue,
here:
> > > >> https://issues.apache.org/jira/browse/OODT). If they were, it
> > > >>would have picked up  your <property .. declaration of filePathSpec
below.
> > > >> It?s been on my TODO list for a long time.
> > > >>
> > > >> Instead I created this:
> > > >>
> > > >> https://issues.apache.org/jira/browse/OODT-639
> > > >>
> > > >>
> > > >> So you can amend your definition below (also note you always have
> > > >>to include  /[Filename] at the end to get the filename you want).
> > > >>
> > > >>   <type id="urn:oodt:EngineeringFile" name="EngineeringFile">
> > > >>     <repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
> > > >>     <versioner
> > > >>
> > class=?org.apache.oodt.cas.filemgr.versioning.ProductTypeMetVersioner?
> > />
> > > >>     <description>The default product type for any kind of
> > > >>file.</description>
> > > >>     <metExtractors>
> > > >>       <extractor
> > > >>
> > >
> > >>class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtrac
> > >>tor">
> > > >>         <configuration>
> > > >>           <!-- you can optionally include the envReplace tag to
> > > >>turn on/off  environment var replacement -->
> > > >>           <property name="nsAware" value="true" />
> > > >>           <property name="elementNs" value="CAS" />
> > > >>           <property name="elements"
> > > >> value="ProductReceivedTime,ProductName,ProductId" />
> > > >>         </configuration>
> > > >>       </extractor>
> > > >>     </metExtractors>
> > > >>     <metadata>
> > > >>  <keyval>
> > > >>     <key>filePathSpec</key>
> > > >>     <val>/[YearDir]/[DoyDir]/[Filename]</val>
> > > >>   </keyval>
> > > >>
> > > >> </metadata>
> > > >>   </type>
> > > >>
> > > >>
> > > >> See if that fixes it!
> > > >>
> > > >> Cheers,
> > > >> Chris
> > > >>
> > > >> ------------------------
> > > >> Chris Mattmann
> > > >> chris.mattmann@gmail.com
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> -----Original Message-----
> > > >> From: "Mallder, Valerie" <Valerie.Mallder@jhuapl.edu>
> > > >> Reply-To: <dev@oodt.apache.org>
> > > >> Date: Thursday, October 2, 2014 at 2:56 PM
> > > >> To: "dev@oodt.apache.org" <dev@oodt.apache.org>
> > > >> Subject: how to use MetadataBasedFileVersioner properly
> > > >>
> > > >> >Hi All,
> > > >> >
> > > >> >I am trying to use the MetadataBasedFileVersioner to store my
> > > >> >files in a subdirectory of the data/archive folder based on the
> > > >> >values of two metadata elements. Has anyone does this before,
> > > >> >and if so, can you give me some hints.
> > > >> >
> > > >> >My files need to be organized in subdirectories by year and day
> > > >> >of
> > > >>year.
> > > >> >So, lets say the file name is 'myfile', the year is 2014, and
> > > >> >the doy of year is 002. Then, the end result that I am looking
> > > >> >for is to have the final location of my file be:
> > > >>'data/archive/ops/eng/2014/002/myfile.
> > > >> >
> > > >> >However, in my log file, the INFO messages indicate that the
> > > >> >generated final location reference is 'data/archive/ops/eng/myfile'.
> > > >> >And it doesn't include the year and day of year at all. And the
> > > >> >incorrect location leads to other errors. So I want to solve
> > > >> >this one first. Has anyone tried to do something like this before?
> > > >> >
> > > >> >Here is what I have done so far:
> > > >> >
> > > >> >Added two new elements to the .met file 'YearDir' and 'DoyDir'.
> > > >> >Added these new elements to the elements.xml file.
> > > >> >Added these new elements to the product in the
> > > >> >product-type-element-map.xml file.
> > > >> >And, I have made changes to the product-types.xml file, but
> > > >> >here's where I am not sure I've done this properly.
> > > >> >
> > > >> >Here's what product type definition looks like:
> > > >> >  <type id="urn:oodt:EngineeringFile" name="EngineeringFile">
> > > >> >    <repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
> > > >> >    <versioner
> > > >>
> > > >>>class="org.apache.oodt.cas.filemgr.versioning.MetadataBasedFileVe
> > > >>>rsio
> > > >>>ner
> > > >>>">
> > > >> >          <property name="filePathSpec" value="/[YearDir]/[DoyDir]/"
> > />
> > > >> >    </versioner>
> > > >> >    <description>The default product type for any kind of
> > > >> >file.</description>
> > > >> >    <metExtractors>
> > > >> >      <extractor
> > > >>
> > >
> > >>>class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor"
> > > >>>>
> > > >> >        <configuration>
> > > >> >          <!-- you can optionally include the envReplace tag
to
> > > >> >turn on/off environment var replacement -->
> > > >> >          <property name="nsAware" value="true" />
> > > >> >          <property name="elementNs" value="CAS" />
> > > >> >          <property name="elements"
> > > >> >value="ProductReceivedTime,ProductName,ProductId" />
> > > >> >        </configuration>
> > > >> >      </extractor>
> > > >> >    </metExtractors>
> > > >> >    <metadata/>
> > > >> >  </type>
> > > >> >
> > > >> >Thanks in advance for any help or ideas you might have!
> > > >> >
> > > >> >Valerie
> > > >> >
> > > >> >
> > > >> >
> > > >> >Valerie A. Mallder
> > > >> >
> > > >> >New Horizons Deputy Mission System Engineer The Johns Hopkins
> > > >> >University/Applied Physics Laboratory
> > > >> >11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
> > > >> >240-228-7846 (Office) 410-504-2233 (Blackberry)
> > > >> >
> > > >>
> > > >
> >
> >
Mime
View raw message