oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Barber <tom.bar...@meteorite.bi>
Subject Re: more info for OODT-751 OPSUI Pages constantly expire
Date Fri, 21 Nov 2014 15:06:37 GMT
Oi Chris, I sent Val a fix! :P

On 21/11/14 14:46, Mattmann, Chris A (3980) wrote:
> Hi Val,
>
> I haven’t forgot about this - just haven’t had a chance to investigate.
> Hope to have some time in the next week.
>
> Cheers,
> Chris
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398)
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: chris.a.mattmann@nasa.gov
> WWW:  http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Associate Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>
>
> -----Original Message-----
> From: <Mallder>, Valerie <Valerie.Mallder@jhuapl.edu>
> Reply-To: "dev@oodt.apache.org" <dev@oodt.apache.org>
> Date: Thursday, November 13, 2014 at 2:10 PM
> To: "dev@oodt.apache.org" <dev@oodt.apache.org>
> Subject: RE: more info for OODT-751 OPSUI Pages constantly expire
>
>> For those of you who are using the pcs-opsui in an operational
>> environment without getting expired pages please tell me what version you
>> are using, the OS you are running it on, and the browser you are using.
>>
>> I need to demonstrate "something" to my project manager to show the
>> usefulness of OODT in the Jedi instrument science data pipeline here at
>> APL. And right now, I have nothing to "show" for my last few months of
>> work. If someone can tell me a configuration using opsui that is working
>> then maybe I can try to mimic that and get something useful up and
>> running. So far, for me, versions 0.6, 0.7 and the current trunk are
>> showing only expired pages.
>>
>> Thanks very much!
>> Val
>>
>>
>>
>> Sent from my iPhone.
>> ________________________________
>> From: Mallder, Valerie <Valerie.Mallder@jhuapl.edu>
>> Sent: Tuesday, November 11, 2014 2:55:12 PM
>> To: dev@oodt.apache.org
>> Subject: more info for OODT-751 OPSUI Pages constantly expire
>>
>> Hi Chris,
>>
>> I know you are working on this, but I wanted to let you know that I tried
>> to use version 0.6 and am having the same problem in version 0.6. But, I
>> can't guarantee that I installed version 0.6 correctly so I would like to
>> run this by you.  Based on your email below here's what I did:
>>
>> 1. Downloaded pcs-opsui-0.76.war from:
>> http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/
>> and saved it in a new folder named $OODT_HOME/bin/opsui
>>
>> 2. Created a script called $OODT_HOME/bin/opsui/runopsui that sets the
>> following variables (this is output from my script):
>> Using OODT_BASE:  
>> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy
>> Using OODT_HOME:  
>> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy
>> Using OODT_TMPDIR:
>> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/temp
>> Using FILEMGR_URL:       http://localhost:9000
>> Using WORKFLOW_URL:      http://localhost:9001
>> Using RESMGR_URL:        http://localhost:9002
>> Using WORKFLOW_HOME:
>> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/workflow
>> Using RESMGR_HOME:
>> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/resmgr
>> Using CRAWLER_HOME:
>> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/crawler
>> Using TOMCAT_HOME:
>> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/tomcat
>> Using PCS_HOME:   
>> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pcs
>> Using PGE_HOME:   
>> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pge
>> Using PGE_JOBS_DIR:
>> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/jobs
>> Using FEI_DROP_DIR:
>> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/telemetr
>> y
>> Using JEDI_L0_DIR:
>> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l0
>> Using JEDI_L2_DIR:
>> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l2
>> Using ARCHIVE_DIR:
>> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive
>> Using BACKUP_DIR: 
>> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/met
>> Using FAILURE_DIR:
>> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/failure
>> Using JEDI_PIPELINE_DIR: /homes/malldva1/working/pipeline
>> Using SNAPSHOT_DIR:
>> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/jobs
>> /snapshot
>>
>> (Then, I followed the steps at the bottom of:
>> https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+OPSUI
>> )
>>
>> 3. Downloaded 
>> http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/webapp/META-
>> INF/context.xml
>> And saved it as $OODT_HOME/bin/opsui/pcs-opsui.xml
>>
>> 4. Edited $OODT_HOME/bin/opsui/pcs-opsui.xml and changed the first line
>> from:
>> <Context path="/pcs-opsui">
>> To
>> <Context path="/pcs-opsui"
>> docBase="[OODT_HOME]/bin/opsui/pcs-opsui-0.6.war">
>>
>> 5.  Killed any process with "tomcat" in its name to ensure that tomcat is
>> not running.
>>
>> 6. Executed the following command to create a symbolic link:
>> cd $OODT_HOME/bin/opsui/
>> ln -s pcs-opsui.xml to $TOMCAT_HOME/conf/Catalina/localhost/pcs-opsui.xml
>>
>> 7. Then, in my script that sets the environment variables, I added the
>> following command to start tomcat.
>> exec "$OODT_BASE"/tomcat/bin/catalina.sh start
>>
>> 8. Then, I ran my new $OODT_HOME/bin/opsui/runopsui script
>>
>> 9. Then, I started firefox  and went to "localhost:8080/pcs-opsui"
>>
>>
>>
>> The home page comes up, and the PCS Status page comes up, but all other
>> pages are expired.  I should note that the "radix" installation of
>> version 0.7 points me to localhost:8080/opsui, while step 10 of the
>> instructions from the quick start guide of the wiki point me to
>> localhost:8080/pcs-opsui. And, no matter which one I go to, the pages
>> expire.
>>
>> I also tried clearing out my cache and browser history and restarting
>> firefox and topcat, etc. and nothing seems to help.
>>
>> If I have gotten the version 0.6 up and running correctly, I thought you
>> would be interested to know that I am seeing the error. If I did not get
>> version 0.6 up and running, then I need more hints on how to do it.  :)
>>
>> Thanks,
>> Val
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> Valerie A. Mallder
>> New Horizons Deputy Mission System Engineer
>> Johns Hopkins University/Applied Physics Laboratory
>>
>>
>>> -----Original Message-----
>>> From: Mattmann, Chris A (3980) [mailto:chris.a.mattmann@jpl.nasa.gov]
>>> Sent: Friday, October 03, 2014 1:07 PM
>>> To: dev@oodt.apache.org
>>> Subject: Re: Success! RE: how to use MetadataBasedFileVersioner properly
>>>
>>> Awesome Val! :)
>>>
>>> I think you¹re running into this:
>>>
>>> https://issues.apache.org/jira/browse/OODT-751
>>>
>>>
>>> In the meanwhile, try the 0.6 OPSUI, which you can grab from here:
>>>
>>> http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/
>>>
>>>
>>> Grab the WAR file and drop it into your favorite container.
>>> Make sure you have all of these environment variables installed
>>> *before* starting Tomcat or Jetty, etc.:
>>>
>>>
>>> https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+OPSU
>>> I
>>>
>>>
>>> (see steps at bottom and replace 0.5 with 0.6)
>>>
>>> I¹m working on a fix for OODT-751, at which point RADIX will be pretty
>>> buff. Next
>>> steps at that point:
>>>
>>> 1. Release 0.7 and then encourage folks to get started by using the
>>> Vagrant build,
>>> e.g.,
>>>
>>> git clone https://github.com/apache/oodt cd vagrant/radix vagrant up
>>>
>>> 2. Fix OODT-491 and remaining workflow manager issues for Wengine 3.
>>> Work
>>> on Streaming OODT API with AMP Stack (via M. Starch et al) and release
>>> in 0.8.
>>> 4. Conquer and win.
>>>
>>> Thanks!
>>>
>>> Cheers,
>>> Chris
>>>
>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> +++++
>>> Chris Mattmann, Ph.D.
>>> Chief Architect
>>> Instrument Software and Science Data Systems Section (398) NASA Jet
>>> Propulsion Laboratory Pasadena, CA 91109 USA
>>> Office: 168-519, Mailstop: 168-527
>>> Email: chris.a.mattmann@nasa.gov
>>> WWW:  http://sunset.usc.edu/~mattmann/
>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> +++++
>>> Adjunct Associate Professor, Computer Science Department University of
>>> Southern California, Los Angeles, CA 90089 USA
>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> +++++
>>>
>>>
>>>
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: <Mallder>, Valerie <Valerie.Mallder@jhuapl.edu>
>>> Reply-To: "dev@oodt.apache.org" <dev@oodt.apache.org>
>>> Date: Friday, October 3, 2014 at 9:53 AM
>>> To: "dev@oodt.apache.org" <dev@oodt.apache.org>
>>> Subject: Success!  RE: how to use MetadataBasedFileVersioner properly
>>>
>>>> Hi Chris,
>>>>
>>>> Yes, that indeed fixed it!  Thanks so much!  I now have 18 engineering
>>>> files ingested. Whoo hoo!!
>>>>
>>>> Ok, so now, I would like to see what the opsui has to say about my
>>>> ingested files. And here comes probably a really stupid question.  I am
>>>> using Firefox on a Redhat Linux box, and I'm not that familiar with
>>>> Firefox and it's settings, so this might be a browser setting issue.
>>>> When I startup the opsui and select "File Catalog Browse" it shows that
>>>> I have
>>>> 18 EngineeringFiles. Then, when I select File Catalog Browse from the
>>>> strip of options under the logo, I get a message saying the page has
>>>> expired. And here's the link that shows up in the address bar:
>>>> http://localhost:8080/opsui/?wicket:interface=:7:fmbrowser_link::ILinkL
>>>> ist
>>>> ener::
>>>>
>>>> And I get the page expired message for all of the options that I
>>> select.
>>>> Any idea's on this one??
>>>>
>>>> In the meantime though, I will start playing with an action for post
>>>> ingestion success that simply makes another copy of all these
>>>> engineering files and puts them another folder for Level 0 files.
>>>>
>>>> Thanks,
>>>> Val
>>>>
>>>>
>>>>
>>>>
>>>> Valerie A. Mallder
>>>> New Horizons Deputy Mission System Engineer Johns Hopkins
>>>> University/Applied Physics Laboratory
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: Chris Mattmann [mailto:chris.mattmann@gmail.com]
>>>>> Sent: Friday, October 03, 2014 3:03 AM
>>>>> To: dev@oodt.apache.org
>>>>> Subject: Re: how to use MetadataBasedFileVersioner properly
>>>>>
>>>>> You?re almost there Val!
>>>>>
>>>>> Unfortunately Versioners right now aren?t configurable from product
>>>>> type policy  (would be great to capture this in a JIRA issue, here:
>>>>> https://issues.apache.org/jira/browse/OODT). If they were, it would
>>>>> have picked up  your <property .. declaration of filePathSpec below.
>>>>> It?s been on my TODO list for a long time.
>>>>>
>>>>> Instead I created this:
>>>>>
>>>>> https://issues.apache.org/jira/browse/OODT-639
>>>>>
>>>>>
>>>>> So you can amend your definition below (also note you always have to
>>>>> include  /[Filename] at the end to get the filename you want).
>>>>>
>>>>>   <type id="urn:oodt:EngineeringFile" name="EngineeringFile">
>>>>>     <repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
>>>>>     <versioner
>>>>>
>>> class=?org.apache.oodt.cas.filemgr.versioning.ProductTypeMetVersioner?/>
>>>>>     <description>The default product type for any kind of
>>>>> file.</description>
>>>>>     <metExtractors>
>>>>>       <extractor
>>>>>
>>>>> class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor
>>>>> ">
>>>>>         <configuration>
>>>>>           <!-- you can optionally include the envReplace tag to turn
>>>>> on/off  environment var replacement -->
>>>>>           <property name="nsAware" value="true" />
>>>>>           <property name="elementNs" value="CAS" />
>>>>>           <property name="elements"
>>>>> value="ProductReceivedTime,ProductName,ProductId" />
>>>>>         </configuration>
>>>>>       </extractor>
>>>>>     </metExtractors>
>>>>>     <metadata>
>>>>>  <keyval>
>>>>>     <key>filePathSpec</key>
>>>>>     <val>/[YearDir]/[DoyDir]/[Filename]</val>
>>>>>   </keyval>
>>>>>
>>>>> </metadata>
>>>>>   </type>
>>>>>
>>>>>
>>>>> See if that fixes it!
>>>>>
>>>>> Cheers,
>>>>> Chris
>>>>>
>>>>> ------------------------
>>>>> Chris Mattmann
>>>>> chris.mattmann@gmail.com
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: "Mallder, Valerie" <Valerie.Mallder@jhuapl.edu>
>>>>> Reply-To: <dev@oodt.apache.org>
>>>>> Date: Thursday, October 2, 2014 at 2:56 PM
>>>>> To: "dev@oodt.apache.org" <dev@oodt.apache.org>
>>>>> Subject: how to use MetadataBasedFileVersioner properly
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> I am trying to use the MetadataBasedFileVersioner to store my files
>>>>>> in a subdirectory of the data/archive folder based on the values
of
>>>>>> two metadata elements. Has anyone does this before, and if so, can
>>>>>> you give me some hints.
>>>>>>
>>>>>> My files need to be organized in subdirectories by year and day of
>>>>> year.
>>>>>> So, lets say the file name is 'myfile', the year is 2014, and the
>>>>>> doy of year is 002. Then, the end result that I am looking for is
to
>>>>>> have the final location of my file be:
>>>>> 'data/archive/ops/eng/2014/002/myfile.
>>>>>> However, in my log file, the INFO messages indicate that the
>>>>>> generated final location reference is 'data/archive/ops/eng/myfile'.
>>>>>> And it doesn't include the year and day of year at all. And the
>>>>>> incorrect location leads to other errors. So I want to solve this
>>>>>> one first. Has anyone tried to do something like this before?
>>>>>>
>>>>>> Here is what I have done so far:
>>>>>>
>>>>>> Added two new elements to the .met file 'YearDir' and 'DoyDir'.
>>>>>> Added these new elements to the elements.xml file.
>>>>>> Added these new elements to the product in the
>>>>>> product-type-element-map.xml file.
>>>>>> And, I have made changes to the product-types.xml file, but here's
>>>>>> where I am not sure I've done this properly.
>>>>>>
>>>>>> Here's what product type definition looks like:
>>>>>>  <type id="urn:oodt:EngineeringFile" name="EngineeringFile">
>>>>>>    <repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
>>>>>>    <versioner
>>>>>> class="org.apache.oodt.cas.filemgr.versioning.MetadataBasedFileVersio
>>>>>> ner
>>>>>> ">
>>>>>>          <property name="filePathSpec"
>>> value="/[YearDir]/[DoyDir]/" />
>>>>>>    </versioner>
>>>>>>    <description>The default product type for any kind of
>>>>>> file.</description>
>>>>>>    <metExtractors>
>>>>>>      <extractor
>>>>>> class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtracto
>>>>>> r"
>>>>>>        <configuration>
>>>>>>          <!-- you can optionally include the envReplace tag to
turn
>>>>>> on/off environment var replacement -->
>>>>>>          <property name="nsAware" value="true" />
>>>>>>          <property name="elementNs" value="CAS" />
>>>>>>          <property name="elements"
>>>>>> value="ProductReceivedTime,ProductName,ProductId" />
>>>>>>        </configuration>
>>>>>>      </extractor>
>>>>>>    </metExtractors>
>>>>>>    <metadata/>
>>>>>>  </type>
>>>>>>
>>>>>> Thanks in advance for any help or ideas you might have!
>>>>>>
>>>>>> Valerie
>>>>>>
>>>>>>
>>>>>>
>>>>>> Valerie A. Mallder
>>>>>>
>>>>>> New Horizons Deputy Mission System Engineer The Johns Hopkins
>>>>>> University/Applied Physics Laboratory
>>>>>> 11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
>>>>>> 240-228-7846 (Office) 410-504-2233 (Blackberry)
>>>>>>


-- 
*Tom Barber* | Technical Director

meteorite bi
*T:* +44 20 8133 3730
*W:* www.meteorite.bi | *Skype:* meteorite.consulting
*A:* Surrey Technology Centre, Surrey Research Park, Guildford, GU2 7YG, UK

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message