oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (3980)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: more info for OODT-751 OPSUI Pages constantly expire
Date Fri, 21 Nov 2014 14:46:06 GMT
Hi Val,

I haven’t forgot about this - just haven’t had a chance to investigate.
Hope to have some time in the next week.

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++






-----Original Message-----
From: <Mallder>, Valerie <Valerie.Mallder@jhuapl.edu>
Reply-To: "dev@oodt.apache.org" <dev@oodt.apache.org>
Date: Thursday, November 13, 2014 at 2:10 PM
To: "dev@oodt.apache.org" <dev@oodt.apache.org>
Subject: RE: more info for OODT-751 OPSUI Pages constantly expire

>For those of you who are using the pcs-opsui in an operational
>environment without getting expired pages please tell me what version you
>are using, the OS you are running it on, and the browser you are using.
>
>I need to demonstrate "something" to my project manager to show the
>usefulness of OODT in the Jedi instrument science data pipeline here at
>APL. And right now, I have nothing to "show" for my last few months of
>work. If someone can tell me a configuration using opsui that is working
>then maybe I can try to mimic that and get something useful up and
>running. So far, for me, versions 0.6, 0.7 and the current trunk are
>showing only expired pages.
>
>Thanks very much!
>Val
>
>
>
>Sent from my iPhone.
>________________________________
>From: Mallder, Valerie <Valerie.Mallder@jhuapl.edu>
>Sent: Tuesday, November 11, 2014 2:55:12 PM
>To: dev@oodt.apache.org
>Subject: more info for OODT-751 OPSUI Pages constantly expire
>
>Hi Chris,
>
>I know you are working on this, but I wanted to let you know that I tried
>to use version 0.6 and am having the same problem in version 0.6. But, I
>can't guarantee that I installed version 0.6 correctly so I would like to
>run this by you.  Based on your email below here's what I did:
>
>1. Downloaded pcs-opsui-0.76.war from:
>http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/
>and saved it in a new folder named $OODT_HOME/bin/opsui
>
>2. Created a script called $OODT_HOME/bin/opsui/runopsui that sets the
>following variables (this is output from my script):
>Using OODT_BASE:  
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy
>Using OODT_HOME:  
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy
>Using OODT_TMPDIR:
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/temp
>Using FILEMGR_URL:       http://localhost:9000
>Using WORKFLOW_URL:      http://localhost:9001
>Using RESMGR_URL:        http://localhost:9002
>Using WORKFLOW_HOME:
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/workflow
>Using RESMGR_HOME:
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/resmgr
>Using CRAWLER_HOME:
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/crawler
>Using TOMCAT_HOME:
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/tomcat
>Using PCS_HOME:   
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pcs
>Using PGE_HOME:   
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pge
>Using PGE_JOBS_DIR:
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/jobs
>Using FEI_DROP_DIR:
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/telemetr
>y
>Using JEDI_L0_DIR:
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l0
>Using JEDI_L2_DIR:
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l2
>Using ARCHIVE_DIR:
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive
>Using BACKUP_DIR: 
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/met
>Using FAILURE_DIR:
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/failure
>Using JEDI_PIPELINE_DIR: /homes/malldva1/working/pipeline
>Using SNAPSHOT_DIR:
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/jobs
>/snapshot
>
>(Then, I followed the steps at the bottom of:
>https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+OPSUI
>)
>
>3. Downloaded 
>http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/webapp/META-
>INF/context.xml
>And saved it as $OODT_HOME/bin/opsui/pcs-opsui.xml
>
>4. Edited $OODT_HOME/bin/opsui/pcs-opsui.xml and changed the first line
>from:
><Context path="/pcs-opsui">
>To
><Context path="/pcs-opsui"
>docBase="[OODT_HOME]/bin/opsui/pcs-opsui-0.6.war">
>
>5.  Killed any process with "tomcat" in its name to ensure that tomcat is
>not running.
>
>6. Executed the following command to create a symbolic link:
>cd $OODT_HOME/bin/opsui/
>ln -s pcs-opsui.xml to $TOMCAT_HOME/conf/Catalina/localhost/pcs-opsui.xml
>
>7. Then, in my script that sets the environment variables, I added the
>following command to start tomcat.
>exec "$OODT_BASE"/tomcat/bin/catalina.sh start
>
>8. Then, I ran my new $OODT_HOME/bin/opsui/runopsui script
>
>9. Then, I started firefox  and went to "localhost:8080/pcs-opsui"
>
>
>
>The home page comes up, and the PCS Status page comes up, but all other
>pages are expired.  I should note that the "radix" installation of
>version 0.7 points me to localhost:8080/opsui, while step 10 of the
>instructions from the quick start guide of the wiki point me to
>localhost:8080/pcs-opsui. And, no matter which one I go to, the pages
>expire.
>
>I also tried clearing out my cache and browser history and restarting
>firefox and topcat, etc. and nothing seems to help.
>
>If I have gotten the version 0.6 up and running correctly, I thought you
>would be interested to know that I am seeing the error. If I did not get
>version 0.6 up and running, then I need more hints on how to do it.  :)
>
>Thanks,
>Val
>
>
>
>
>
>
>
>
>
>Valerie A. Mallder
>New Horizons Deputy Mission System Engineer
>Johns Hopkins University/Applied Physics Laboratory
>
>
>> -----Original Message-----
>> From: Mattmann, Chris A (3980) [mailto:chris.a.mattmann@jpl.nasa.gov]
>> Sent: Friday, October 03, 2014 1:07 PM
>> To: dev@oodt.apache.org
>> Subject: Re: Success! RE: how to use MetadataBasedFileVersioner properly
>>
>> Awesome Val! :)
>>
>> I think you¹re running into this:
>>
>> https://issues.apache.org/jira/browse/OODT-751
>>
>>
>> In the meanwhile, try the 0.6 OPSUI, which you can grab from here:
>>
>> http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/
>>
>>
>> Grab the WAR file and drop it into your favorite container.
>> Make sure you have all of these environment variables installed
>> *before* starting Tomcat or Jetty, etc.:
>>
>> 
>>https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+OPSU
>>I
>>
>>
>> (see steps at bottom and replace 0.5 with 0.6)
>>
>> I¹m working on a fix for OODT-751, at which point RADIX will be pretty
>>buff. Next
>> steps at that point:
>>
>> 1. Release 0.7 and then encourage folks to get started by using the
>>Vagrant build,
>> e.g.,
>>
>> git clone https://github.com/apache/oodt cd vagrant/radix vagrant up
>>
>> 2. Fix OODT-491 and remaining workflow manager issues for Wengine 3.
>>Work
>> on Streaming OODT API with AMP Stack (via M. Starch et al) and release
>>in 0.8.
>> 4. Conquer and win.
>>
>> Thanks!
>>
>> Cheers,
>> Chris
>>
>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> +++++
>> Chris Mattmann, Ph.D.
>> Chief Architect
>> Instrument Software and Science Data Systems Section (398) NASA Jet
>> Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 168-519, Mailstop: 168-527
>> Email: chris.a.mattmann@nasa.gov
>> WWW:  http://sunset.usc.edu/~mattmann/
>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> +++++
>> Adjunct Associate Professor, Computer Science Department University of
>> Southern California, Los Angeles, CA 90089 USA
>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> +++++
>>
>>
>>
>>
>>
>>
>> -----Original Message-----
>> From: <Mallder>, Valerie <Valerie.Mallder@jhuapl.edu>
>> Reply-To: "dev@oodt.apache.org" <dev@oodt.apache.org>
>> Date: Friday, October 3, 2014 at 9:53 AM
>> To: "dev@oodt.apache.org" <dev@oodt.apache.org>
>> Subject: Success!  RE: how to use MetadataBasedFileVersioner properly
>>
>> >Hi Chris,
>> >
>> >Yes, that indeed fixed it!  Thanks so much!  I now have 18 engineering
>> >files ingested. Whoo hoo!!
>> >
>> >Ok, so now, I would like to see what the opsui has to say about my
>> >ingested files. And here comes probably a really stupid question.  I am
>> >using Firefox on a Redhat Linux box, and I'm not that familiar with
>> >Firefox and it's settings, so this might be a browser setting issue.
>> >When I startup the opsui and select "File Catalog Browse" it shows that
>> >I have
>> >18 EngineeringFiles. Then, when I select File Catalog Browse from the
>> >strip of options under the logo, I get a message saying the page has
>> >expired. And here's the link that shows up in the address bar:
>> >http://localhost:8080/opsui/?wicket:interface=:7:fmbrowser_link::ILinkL
>> >ist
>> >ener::
>> >
>> >And I get the page expired message for all of the options that I
>>select.
>> >Any idea's on this one??
>> >
>> >In the meantime though, I will start playing with an action for post
>> >ingestion success that simply makes another copy of all these
>> >engineering files and puts them another folder for Level 0 files.
>> >
>> >Thanks,
>> >Val
>> >
>> >
>> >
>> >
>> >Valerie A. Mallder
>> >New Horizons Deputy Mission System Engineer Johns Hopkins
>> >University/Applied Physics Laboratory
>> >
>> >
>> >> -----Original Message-----
>> >> From: Chris Mattmann [mailto:chris.mattmann@gmail.com]
>> >> Sent: Friday, October 03, 2014 3:03 AM
>> >> To: dev@oodt.apache.org
>> >> Subject: Re: how to use MetadataBasedFileVersioner properly
>> >>
>> >> You?re almost there Val!
>> >>
>> >> Unfortunately Versioners right now aren?t configurable from product
>> >>type policy  (would be great to capture this in a JIRA issue, here:
>> >> https://issues.apache.org/jira/browse/OODT). If they were, it would
>> >>have picked up  your <property .. declaration of filePathSpec below.
>> >> It?s been on my TODO list for a long time.
>> >>
>> >> Instead I created this:
>> >>
>> >> https://issues.apache.org/jira/browse/OODT-639
>> >>
>> >>
>> >> So you can amend your definition below (also note you always have to
>> >>include  /[Filename] at the end to get the filename you want).
>> >>
>> >>   <type id="urn:oodt:EngineeringFile" name="EngineeringFile">
>> >>     <repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
>> >>     <versioner
>> >> 
>>class=?org.apache.oodt.cas.filemgr.versioning.ProductTypeMetVersioner?/>
>> >>     <description>The default product type for any kind of
>> >>file.</description>
>> >>     <metExtractors>
>> >>       <extractor
>> >>
>> 
>>>>class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor
>>>>">
>> >>         <configuration>
>> >>           <!-- you can optionally include the envReplace tag to turn
>> >>on/off  environment var replacement -->
>> >>           <property name="nsAware" value="true" />
>> >>           <property name="elementNs" value="CAS" />
>> >>           <property name="elements"
>> >> value="ProductReceivedTime,ProductName,ProductId" />
>> >>         </configuration>
>> >>       </extractor>
>> >>     </metExtractors>
>> >>     <metadata>
>> >>  <keyval>
>> >>     <key>filePathSpec</key>
>> >>     <val>/[YearDir]/[DoyDir]/[Filename]</val>
>> >>   </keyval>
>> >>
>> >> </metadata>
>> >>   </type>
>> >>
>> >>
>> >> See if that fixes it!
>> >>
>> >> Cheers,
>> >> Chris
>> >>
>> >> ------------------------
>> >> Chris Mattmann
>> >> chris.mattmann@gmail.com
>> >>
>> >>
>> >>
>> >>
>> >> -----Original Message-----
>> >> From: "Mallder, Valerie" <Valerie.Mallder@jhuapl.edu>
>> >> Reply-To: <dev@oodt.apache.org>
>> >> Date: Thursday, October 2, 2014 at 2:56 PM
>> >> To: "dev@oodt.apache.org" <dev@oodt.apache.org>
>> >> Subject: how to use MetadataBasedFileVersioner properly
>> >>
>> >> >Hi All,
>> >> >
>> >> >I am trying to use the MetadataBasedFileVersioner to store my files
>> >> >in a subdirectory of the data/archive folder based on the values of
>> >> >two metadata elements. Has anyone does this before, and if so, can
>> >> >you give me some hints.
>> >> >
>> >> >My files need to be organized in subdirectories by year and day of
>> >>year.
>> >> >So, lets say the file name is 'myfile', the year is 2014, and the
>> >> >doy of year is 002. Then, the end result that I am looking for is to
>> >> >have the final location of my file be:
>> >>'data/archive/ops/eng/2014/002/myfile.
>> >> >
>> >> >However, in my log file, the INFO messages indicate that the
>> >> >generated final location reference is 'data/archive/ops/eng/myfile'.
>> >> >And it doesn't include the year and day of year at all. And the
>> >> >incorrect location leads to other errors. So I want to solve this
>> >> >one first. Has anyone tried to do something like this before?
>> >> >
>> >> >Here is what I have done so far:
>> >> >
>> >> >Added two new elements to the .met file 'YearDir' and 'DoyDir'.
>> >> >Added these new elements to the elements.xml file.
>> >> >Added these new elements to the product in the
>> >> >product-type-element-map.xml file.
>> >> >And, I have made changes to the product-types.xml file, but here's
>> >> >where I am not sure I've done this properly.
>> >> >
>> >> >Here's what product type definition looks like:
>> >> >  <type id="urn:oodt:EngineeringFile" name="EngineeringFile">
>> >> >    <repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
>> >> >    <versioner
>> >>
>> >>>class="org.apache.oodt.cas.filemgr.versioning.MetadataBasedFileVersio
>> >>>ner
>> >>>">
>> >> >          <property name="filePathSpec"
>>value="/[YearDir]/[DoyDir]/" />
>> >> >    </versioner>
>> >> >    <description>The default product type for any kind of
>> >> >file.</description>
>> >> >    <metExtractors>
>> >> >      <extractor
>> >>
>> 
>>>>>class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtracto
>>>>>r"
>> >>>>
>> >> >        <configuration>
>> >> >          <!-- you can optionally include the envReplace tag to turn
>> >> >on/off environment var replacement -->
>> >> >          <property name="nsAware" value="true" />
>> >> >          <property name="elementNs" value="CAS" />
>> >> >          <property name="elements"
>> >> >value="ProductReceivedTime,ProductName,ProductId" />
>> >> >        </configuration>
>> >> >      </extractor>
>> >> >    </metExtractors>
>> >> >    <metadata/>
>> >> >  </type>
>> >> >
>> >> >Thanks in advance for any help or ideas you might have!
>> >> >
>> >> >Valerie
>> >> >
>> >> >
>> >> >
>> >> >Valerie A. Mallder
>> >> >
>> >> >New Horizons Deputy Mission System Engineer The Johns Hopkins
>> >> >University/Applied Physics Laboratory
>> >> >11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
>> >> >240-228-7846 (Office) 410-504-2233 (Blackberry)
>> >> >
>> >>
>> >
>

Mime
View raw message