oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Barber <tom.bar...@meteorite.bi>
Subject Re: more info for OODT-751 OPSUI Pages constantly expire
Date Thu, 13 Nov 2014 15:40:36 GMT
Hi Valerie,

Is this with or without Radix?

Tom

On 13/11/14 13:10, Mallder, Valerie wrote:
> For those of you who are using the pcs-opsui in an operational environment without getting
expired pages please tell me what version you are using, the OS you are running it on, and
the browser you are using.
>
> I need to demonstrate "something" to my project manager to show the usefulness of OODT
in the Jedi instrument science data pipeline here at APL. And right now, I have nothing to
"show" for my last few months of work. If someone can tell me a configuration using opsui
that is working then maybe I can try to mimic that and get something useful up and running.
So far, for me, versions 0.6, 0.7 and the current trunk are showing only expired pages.
>
> Thanks very much!
> Val
>
>
>
> Sent from my iPhone.
> ________________________________
> From: Mallder, Valerie <Valerie.Mallder@jhuapl.edu>
> Sent: Tuesday, November 11, 2014 2:55:12 PM
> To: dev@oodt.apache.org
> Subject: more info for OODT-751 OPSUI Pages constantly expire
>
> Hi Chris,
>
> I know you are working on this, but I wanted to let you know that I tried to use version
0.6 and am having the same problem in version 0.6. But, I can't guarantee that I installed
version 0.6 correctly so I would like to run this by you.  Based on your email below here's
what I did:
>
> 1. Downloaded pcs-opsui-0.76.war from:
> http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/
> and saved it in a new folder named $OODT_HOME/bin/opsui
>
> 2. Created a script called $OODT_HOME/bin/opsui/runopsui that sets the following variables
(this is output from my script):
> Using OODT_BASE:         /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy
> Using OODT_HOME:         /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy
> Using OODT_TMPDIR:       /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/temp
> Using FILEMGR_URL:       http://localhost:9000
> Using WORKFLOW_URL:      http://localhost:9001
> Using RESMGR_URL:        http://localhost:9002
> Using WORKFLOW_HOME:     /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/workflow
> Using RESMGR_HOME:       /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/resmgr
> Using CRAWLER_HOME:      /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/crawler
> Using TOMCAT_HOME:       /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/tomcat
> Using PCS_HOME:          /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pcs
> Using PGE_HOME:          /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pge
> Using PGE_JOBS_DIR:      /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/jobs
> Using FEI_DROP_DIR:      /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/telemetry
> Using JEDI_L0_DIR:       /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l0
> Using JEDI_L2_DIR:       /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l2
> Using ARCHIVE_DIR:       /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive
> Using BACKUP_DIR:        /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/met
> Using FAILURE_DIR:       /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/failure
> Using JEDI_PIPELINE_DIR: /homes/malldva1/working/pipeline
> Using SNAPSHOT_DIR:      /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/jobs/snapshot
>
> (Then, I followed the steps at the bottom of: https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+OPSUI)
>
> 3. Downloaded http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/webapp/META-INF/context.xml
> And saved it as $OODT_HOME/bin/opsui/pcs-opsui.xml
>
> 4. Edited $OODT_HOME/bin/opsui/pcs-opsui.xml and changed the first line from:
> <Context path="/pcs-opsui">
> To
> <Context path="/pcs-opsui" docBase="[OODT_HOME]/bin/opsui/pcs-opsui-0.6.war">
>
> 5.  Killed any process with "tomcat" in its name to ensure that tomcat is not running.
>
> 6. Executed the following command to create a symbolic link:
> cd $OODT_HOME/bin/opsui/
> ln -s pcs-opsui.xml to $TOMCAT_HOME/conf/Catalina/localhost/pcs-opsui.xml
>
> 7. Then, in my script that sets the environment variables, I added the following command
to start tomcat.
> exec "$OODT_BASE"/tomcat/bin/catalina.sh start
>
> 8. Then, I ran my new $OODT_HOME/bin/opsui/runopsui script
>
> 9. Then, I started firefox  and went to "localhost:8080/pcs-opsui"
>
>
>
> The home page comes up, and the PCS Status page comes up, but all other pages are expired.
 I should note that the "radix" installation of version 0.7 points me to localhost:8080/opsui,
while step 10 of the instructions from the quick start guide of the wiki point me to localhost:8080/pcs-opsui.
And, no matter which one I go to, the pages expire.
>
> I also tried clearing out my cache and browser history and restarting firefox and topcat,
etc. and nothing seems to help.
>
> If I have gotten the version 0.6 up and running correctly, I thought you would be interested
to know that I am seeing the error. If I did not get version 0.6 up and running, then I need
more hints on how to do it.  :)
>
> Thanks,
> Val
>
>
>
>
>
>
>
>
>
> Valerie A. Mallder
> New Horizons Deputy Mission System Engineer
> Johns Hopkins University/Applied Physics Laboratory
>
>
>> -----Original Message-----
>> From: Mattmann, Chris A (3980) [mailto:chris.a.mattmann@jpl.nasa.gov]
>> Sent: Friday, October 03, 2014 1:07 PM
>> To: dev@oodt.apache.org
>> Subject: Re: Success! RE: how to use MetadataBasedFileVersioner properly
>>
>> Awesome Val! :)
>>
>> I think you¹re running into this:
>>
>> https://issues.apache.org/jira/browse/OODT-751
>>
>>
>> In the meanwhile, try the 0.6 OPSUI, which you can grab from here:
>>
>> http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/
>>
>>
>> Grab the WAR file and drop it into your favorite container.
>> Make sure you have all of these environment variables installed
>> *before* starting Tomcat or Jetty, etc.:
>>
>> https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+OPSUI
>>
>>
>> (see steps at bottom and replace 0.5 with 0.6)
>>
>> I¹m working on a fix for OODT-751, at which point RADIX will be pretty buff. Next
>> steps at that point:
>>
>> 1. Release 0.7 and then encourage folks to get started by using the Vagrant build,
>> e.g.,
>>
>> git clone https://github.com/apache/oodt cd vagrant/radix vagrant up
>>
>> 2. Fix OODT-491 and remaining workflow manager issues for Wengine 3. Work
>> on Streaming OODT API with AMP Stack (via M. Starch et al) and release in 0.8.
>> 4. Conquer and win.
>>
>> Thanks!
>>
>> Cheers,
>> Chris
>>
>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> +++++
>> Chris Mattmann, Ph.D.
>> Chief Architect
>> Instrument Software and Science Data Systems Section (398) NASA Jet
>> Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 168-519, Mailstop: 168-527
>> Email: chris.a.mattmann@nasa.gov
>> WWW:  http://sunset.usc.edu/~mattmann/
>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> +++++
>> Adjunct Associate Professor, Computer Science Department University of
>> Southern California, Los Angeles, CA 90089 USA
>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> +++++
>>
>>
>>
>>
>>
>>
>> -----Original Message-----
>> From: <Mallder>, Valerie <Valerie.Mallder@jhuapl.edu>
>> Reply-To: "dev@oodt.apache.org" <dev@oodt.apache.org>
>> Date: Friday, October 3, 2014 at 9:53 AM
>> To: "dev@oodt.apache.org" <dev@oodt.apache.org>
>> Subject: Success!  RE: how to use MetadataBasedFileVersioner properly
>>
>>> Hi Chris,
>>>
>>> Yes, that indeed fixed it!  Thanks so much!  I now have 18 engineering
>>> files ingested. Whoo hoo!!
>>>
>>> Ok, so now, I would like to see what the opsui has to say about my
>>> ingested files. And here comes probably a really stupid question.  I am
>>> using Firefox on a Redhat Linux box, and I'm not that familiar with
>>> Firefox and it's settings, so this might be a browser setting issue.
>>> When I startup the opsui and select "File Catalog Browse" it shows that
>>> I have
>>> 18 EngineeringFiles. Then, when I select File Catalog Browse from the
>>> strip of options under the logo, I get a message saying the page has
>>> expired. And here's the link that shows up in the address bar:
>>> http://localhost:8080/opsui/?wicket:interface=:7:fmbrowser_link::ILinkL
>>> ist
>>> ener::
>>>
>>> And I get the page expired message for all of the options that I select.
>>> Any idea's on this one??
>>>
>>> In the meantime though, I will start playing with an action for post
>>> ingestion success that simply makes another copy of all these
>>> engineering files and puts them another folder for Level 0 files.
>>>
>>> Thanks,
>>> Val
>>>
>>>
>>>
>>>
>>> Valerie A. Mallder
>>> New Horizons Deputy Mission System Engineer Johns Hopkins
>>> University/Applied Physics Laboratory
>>>
>>>
>>>> -----Original Message-----
>>>> From: Chris Mattmann [mailto:chris.mattmann@gmail.com]
>>>> Sent: Friday, October 03, 2014 3:03 AM
>>>> To: dev@oodt.apache.org
>>>> Subject: Re: how to use MetadataBasedFileVersioner properly
>>>>
>>>> You?re almost there Val!
>>>>
>>>> Unfortunately Versioners right now aren?t configurable from product
>>>> type policy  (would be great to capture this in a JIRA issue, here:
>>>> https://issues.apache.org/jira/browse/OODT). If they were, it would
>>>> have picked up  your <property .. declaration of filePathSpec below.
>>>> It?s been on my TODO list for a long time.
>>>>
>>>> Instead I created this:
>>>>
>>>> https://issues.apache.org/jira/browse/OODT-639
>>>>
>>>>
>>>> So you can amend your definition below (also note you always have to
>>>> include  /[Filename] at the end to get the filename you want).
>>>>
>>>>    <type id="urn:oodt:EngineeringFile" name="EngineeringFile">
>>>>      <repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
>>>>      <versioner
>>>> class=?org.apache.oodt.cas.filemgr.versioning.ProductTypeMetVersioner?/>
>>>>      <description>The default product type for any kind of
>>>> file.</description>
>>>>      <metExtractors>
>>>>        <extractor
>>>>
>>>> class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor">
>>>>          <configuration>
>>>>            <!-- you can optionally include the envReplace tag to turn
>>>> on/off  environment var replacement -->
>>>>            <property name="nsAware" value="true" />
>>>>            <property name="elementNs" value="CAS" />
>>>>            <property name="elements"
>>>> value="ProductReceivedTime,ProductName,ProductId" />
>>>>          </configuration>
>>>>        </extractor>
>>>>      </metExtractors>
>>>>      <metadata>
>>>>   <keyval>
>>>>      <key>filePathSpec</key>
>>>>      <val>/[YearDir]/[DoyDir]/[Filename]</val>
>>>>    </keyval>
>>>>
>>>> </metadata>
>>>>    </type>
>>>>
>>>>
>>>> See if that fixes it!
>>>>
>>>> Cheers,
>>>> Chris
>>>>
>>>> ------------------------
>>>> Chris Mattmann
>>>> chris.mattmann@gmail.com
>>>>
>>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: "Mallder, Valerie" <Valerie.Mallder@jhuapl.edu>
>>>> Reply-To: <dev@oodt.apache.org>
>>>> Date: Thursday, October 2, 2014 at 2:56 PM
>>>> To: "dev@oodt.apache.org" <dev@oodt.apache.org>
>>>> Subject: how to use MetadataBasedFileVersioner properly
>>>>
>>>>> Hi All,
>>>>>
>>>>> I am trying to use the MetadataBasedFileVersioner to store my files
>>>>> in a subdirectory of the data/archive folder based on the values of
>>>>> two metadata elements. Has anyone does this before, and if so, can
>>>>> you give me some hints.
>>>>>
>>>>> My files need to be organized in subdirectories by year and day of
>>>> year.
>>>>> So, lets say the file name is 'myfile', the year is 2014, and the
>>>>> doy of year is 002. Then, the end result that I am looking for is to
>>>>> have the final location of my file be:
>>>> 'data/archive/ops/eng/2014/002/myfile.
>>>>> However, in my log file, the INFO messages indicate that the
>>>>> generated final location reference is 'data/archive/ops/eng/myfile'.
>>>>> And it doesn't include the year and day of year at all. And the
>>>>> incorrect location leads to other errors. So I want to solve this
>>>>> one first. Has anyone tried to do something like this before?
>>>>>
>>>>> Here is what I have done so far:
>>>>>
>>>>> Added two new elements to the .met file 'YearDir' and 'DoyDir'.
>>>>> Added these new elements to the elements.xml file.
>>>>> Added these new elements to the product in the
>>>>> product-type-element-map.xml file.
>>>>> And, I have made changes to the product-types.xml file, but here's
>>>>> where I am not sure I've done this properly.
>>>>>
>>>>> Here's what product type definition looks like:
>>>>>   <type id="urn:oodt:EngineeringFile" name="EngineeringFile">
>>>>>     <repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
>>>>>     <versioner
>>>>> class="org.apache.oodt.cas.filemgr.versioning.MetadataBasedFileVersio
>>>>> ner
>>>>> ">
>>>>>           <property name="filePathSpec" value="/[YearDir]/[DoyDir]/"
/>
>>>>>     </versioner>
>>>>>     <description>The default product type for any kind of
>>>>> file.</description>
>>>>>     <metExtractors>
>>>>>       <extractor
>>>>> class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor"
>>>>>         <configuration>
>>>>>           <!-- you can optionally include the envReplace tag to turn
>>>>> on/off environment var replacement -->
>>>>>           <property name="nsAware" value="true" />
>>>>>           <property name="elementNs" value="CAS" />
>>>>>           <property name="elements"
>>>>> value="ProductReceivedTime,ProductName,ProductId" />
>>>>>         </configuration>
>>>>>       </extractor>
>>>>>     </metExtractors>
>>>>>     <metadata/>
>>>>>   </type>
>>>>>
>>>>> Thanks in advance for any help or ideas you might have!
>>>>>
>>>>> Valerie
>>>>>
>>>>>
>>>>>
>>>>> Valerie A. Mallder
>>>>>
>>>>> New Horizons Deputy Mission System Engineer The Johns Hopkins
>>>>> University/Applied Physics Laboratory
>>>>> 11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
>>>>> 240-228-7846 (Office) 410-504-2233 (Blackberry)
>>>>>
>


-- 
*Tom Barber* | Technical Director

meteorite bi
*T:* +44 20 8133 3730
*W:* www.meteorite.bi | *Skype:* meteorite.consulting
*A:* Surrey Technology Centre, Surrey Research Park, Guildford, GU2 7YG, UK

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message