oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Barber <tom.bar...@meteorite.bi>
Subject Re: more info for OODT-751 OPSUI Pages constantly expire
Date Fri, 14 Nov 2014 01:39:21 GMT
Hi Val,

I tried a 0.7 OpsUI over 0.6 filemgr to see if that made a difference 
but it really doesn't like that.

I can probably have a prod about over the weekend to see if I can figure 
out whats knackered.

Don't despair, we'll get you your demo! ;)

Tom


On 14/11/14 01:30, Tom Barber wrote:
> Okay well, I have some good news and some bad news (and its 1:30 am so 
> I'm off to bed in a moment, so this is a one shot email :) )
>
> I have a stock 0.7 radix install with nothing in the filemanager and I 
> see what you see, its always been pretty picky and needs replacing 
> with something better, I keep prodding people about that, it will 
> happen one day ;)
>
> For some reason Wicket has decided a file that has not changed in 4 
> years, now is the worst thing on the planet and it refuses to change 
> the Product information into something the browser can render.
> Caused by: java.io.NotSerializableException: 
> org.apache.oodt.cas.filemgr.structs.ProductType
>
> My 0.7 install has no products in it.
>
> But, I have a 0.6 Radix install that a used for ApacheCon last year, I 
> haven't done much too it apart from put some products into the file 
> manager, and that works fine(mostly, some of the OpsUI stuff is just 
> endlessly broken).
>
> http://ibin.co/1h9uVtTs0XuX
> http://ibin.co/1h9uge8eqLkU
> http://ibin.co/1h9uowI0RaDO
>
> I've not changed anything in the old one, it just uses the GenericFile 
> type, so it does work. Just not all the time.
>
> Out of curiosity have you got any products ingested or are you running 
> on an empty repo?
>
> Tom
>
>
>
> On 14/11/14 00:52, Mallder, Valerie wrote:
>> Hi Tom,
>>
>> I am running with the Radix installation of version 0.7.  Although, when I tried
running with version 0.6 opsui I did not do a full reinstall of the other oodt components
that come with Radix too.
>>
>> Val
>>
>>
>>
>> Sent from my iPhone.
>> ________________________________
>> From: Tom Barber<tom.barber@meteorite.bi>
>> Sent: Thursday, November 13, 2014 10:40:36 AM
>> To:dev@oodt.apache.org
>> Subject: Re: more info for OODT-751 OPSUI Pages constantly expire
>>
>> Hi Valerie,
>>
>> Is this with or without Radix?
>>
>> Tom
>>
>> On 13/11/14 13:10, Mallder, Valerie wrote:
>>> For those of you who are using the pcs-opsui in an operational environment without
getting expired pages please tell me what version you are using, the OS you are running it
on, and the browser you are using.
>>>
>>> I need to demonstrate "something" to my project manager to show the usefulness
of OODT in the Jedi instrument science data pipeline here at APL. And right now, I have nothing
to "show" for my last few months of work. If someone can tell me a configuration using opsui
that is working then maybe I can try to mimic that and get something useful up and running.
So far, for me, versions 0.6, 0.7 and the current trunk are showing only expired pages.
>>>
>>> Thanks very much!
>>> Val
>>>
>>>
>>>
>>> Sent from my iPhone.
>>> ________________________________
>>> From: Mallder, Valerie<Valerie.Mallder@jhuapl.edu>
>>> Sent: Tuesday, November 11, 2014 2:55:12 PM
>>> To:dev@oodt.apache.org
>>> Subject: more info for OODT-751 OPSUI Pages constantly expire
>>>
>>> Hi Chris,
>>>
>>> I know you are working on this, but I wanted to let you know that I tried to
use version 0.6 and am having the same problem in version 0.6. But, I can't guarantee that
I installed version 0.6 correctly so I would like to run this by you.  Based on your email
below here's what I did:
>>>
>>> 1. Downloaded pcs-opsui-0.76.war from:
>>> http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/
>>> and saved it in a new folder named $OODT_HOME/bin/opsui
>>>
>>> 2. Created a script called $OODT_HOME/bin/opsui/runopsui that sets the following
variables (this is output from my script):
>>> Using OODT_BASE:         /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy
>>> Using OODT_HOME:         /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy
>>> Using OODT_TMPDIR:       /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/temp
>>> Using FILEMGR_URL:http://localhost:9000
>>> Using WORKFLOW_URL:http://localhost:9001
>>> Using RESMGR_URL:http://localhost:9002
>>> Using WORKFLOW_HOME:     /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/workflow
>>> Using RESMGR_HOME:       /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/resmgr
>>> Using CRAWLER_HOME:      /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/crawler
>>> Using TOMCAT_HOME:       /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/tomcat
>>> Using PCS_HOME:          /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pcs
>>> Using PGE_HOME:          /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pge
>>> Using PGE_JOBS_DIR:      /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/jobs
>>> Using FEI_DROP_DIR:      /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/telemetry
>>> Using JEDI_L0_DIR:       /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l0
>>> Using JEDI_L2_DIR:       /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l2
>>> Using ARCHIVE_DIR:       /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive
>>> Using BACKUP_DIR:        /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/met
>>> Using FAILURE_DIR:       /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/failure
>>> Using JEDI_PIPELINE_DIR: /homes/malldva1/working/pipeline
>>> Using SNAPSHOT_DIR:      /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/jobs/snapshot
>>>
>>> (Then, I followed the steps at the bottom of:https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+OPSUI)
>>>
>>> 3. Downloadedhttp://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/webapp/META-INF/context.xml
>>> And saved it as $OODT_HOME/bin/opsui/pcs-opsui.xml
>>>
>>> 4. Edited $OODT_HOME/bin/opsui/pcs-opsui.xml and changed the first line from:
>>> <Context path="/pcs-opsui">
>>> To
>>> <Context path="/pcs-opsui" docBase="[OODT_HOME]/bin/opsui/pcs-opsui-0.6.war">
>>>
>>> 5.  Killed any process with "tomcat" in its name to ensure that tomcat is not
running.
>>>
>>> 6. Executed the following command to create a symbolic link:
>>> cd $OODT_HOME/bin/opsui/
>>> ln -s pcs-opsui.xml to $TOMCAT_HOME/conf/Catalina/localhost/pcs-opsui.xml
>>>
>>> 7. Then, in my script that sets the environment variables, I added the following
command to start tomcat.
>>> exec "$OODT_BASE"/tomcat/bin/catalina.sh start
>>>
>>> 8. Then, I ran my new $OODT_HOME/bin/opsui/runopsui script
>>>
>>> 9. Then, I started firefox  and went to "localhost:8080/pcs-opsui"
>>>
>>>
>>>
>>> The home page comes up, and the PCS Status page comes up, but all other pages
are expired.  I should note that the "radix" installation of version 0.7 points me to localhost:8080/opsui,
while step 10 of the instructions from the quick start guide of the wiki point me to localhost:8080/pcs-opsui.
And, no matter which one I go to, the pages expire.
>>>
>>> I also tried clearing out my cache and browser history and restarting firefox
and topcat, etc. and nothing seems to help.
>>>
>>> If I have gotten the version 0.6 up and running correctly, I thought you would
be interested to know that I am seeing the error. If I did not get version 0.6 up and running,
then I need more hints on how to do it.  :)
>>>
>>> Thanks,
>>> Val
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Valerie A. Mallder
>>> New Horizons Deputy Mission System Engineer
>>> Johns Hopkins University/Applied Physics Laboratory
>>>
>>>
>>>> -----Original Message-----
>>>> From: Mattmann, Chris A (3980) [mailto:chris.a.mattmann@jpl.nasa.gov]
>>>> Sent: Friday, October 03, 2014 1:07 PM
>>>> To:dev@oodt.apache.org
>>>> Subject: Re: Success! RE: how to use MetadataBasedFileVersioner properly
>>>>
>>>> Awesome Val! :)
>>>>
>>>> I think you¹re running into this:
>>>>
>>>> https://issues.apache.org/jira/browse/OODT-751
>>>>
>>>>
>>>> In the meanwhile, try the 0.6 OPSUI, which you can grab from here:
>>>>
>>>> http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/
>>>>
>>>>
>>>> Grab the WAR file and drop it into your favorite container.
>>>> Make sure you have all of these environment variables installed
>>>> *before* starting Tomcat or Jetty, etc.:
>>>>
>>>> https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+OPSUI
>>>>
>>>>
>>>> (see steps at bottom and replace 0.5 with 0.6)
>>>>
>>>> I¹m working on a fix for OODT-751, at which point RADIX will be pretty buff.
Next
>>>> steps at that point:
>>>>
>>>> 1. Release 0.7 and then encourage folks to get started by using the Vagrant
build,
>>>> e.g.,
>>>>
>>>> git clonehttps://github.com/apache/oodt  cd vagrant/radix vagrant up
>>>>
>>>> 2. Fix OODT-491 and remaining workflow manager issues for Wengine 3. Work
>>>> on Streaming OODT API with AMP Stack (via M. Starch et al) and release in
0.8.
>>>> 4. Conquer and win.
>>>>
>>>> Thanks!
>>>>
>>>> Cheers,
>>>> Chris
>>>>
>>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> +++++
>>>> Chris Mattmann, Ph.D.
>>>> Chief Architect
>>>> Instrument Software and Science Data Systems Section (398) NASA Jet
>>>> Propulsion Laboratory Pasadena, CA 91109 USA
>>>> Office: 168-519, Mailstop: 168-527
>>>> Email:chris.a.mattmann@nasa.gov
>>>> WWW:http://sunset.usc.edu/~mattmann/
>>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> +++++
>>>> Adjunct Associate Professor, Computer Science Department University of
>>>> Southern California, Los Angeles, CA 90089 USA
>>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> +++++
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: <Mallder>, Valerie<Valerie.Mallder@jhuapl.edu>
>>>> Reply-To:"dev@oodt.apache.org"  <dev@oodt.apache.org>
>>>> Date: Friday, October 3, 2014 at 9:53 AM
>>>> To:"dev@oodt.apache.org"  <dev@oodt.apache.org>
>>>> Subject: Success!  RE: how to use MetadataBasedFileVersioner properly
>>>>
>>>>> Hi Chris,
>>>>>
>>>>> Yes, that indeed fixed it!  Thanks so much!  I now have 18 engineering
>>>>> files ingested. Whoo hoo!!
>>>>>
>>>>> Ok, so now, I would like to see what the opsui has to say about my
>>>>> ingested files. And here comes probably a really stupid question.  I
am
>>>>> using Firefox on a Redhat Linux box, and I'm not that familiar with
>>>>> Firefox and it's settings, so this might be a browser setting issue.
>>>>> When I startup the opsui and select "File Catalog Browse" it shows that
>>>>> I have
>>>>> 18 EngineeringFiles. Then, when I select File Catalog Browse from the
>>>>> strip of options under the logo, I get a message saying the page has
>>>>> expired. And here's the link that shows up in the address bar:
>>>>> http://localhost:8080/opsui/?wicket:interface=:7:fmbrowser_link::ILinkL
>>>>> ist
>>>>> ener::
>>>>>
>>>>> And I get the page expired message for all of the options that I select.
>>>>> Any idea's on this one??
>>>>>
>>>>> In the meantime though, I will start playing with an action for post
>>>>> ingestion success that simply makes another copy of all these
>>>>> engineering files and puts them another folder for Level 0 files.
>>>>>
>>>>> Thanks,
>>>>> Val
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Valerie A. Mallder
>>>>> New Horizons Deputy Mission System Engineer Johns Hopkins
>>>>> University/Applied Physics Laboratory
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Chris Mattmann [mailto:chris.mattmann@gmail.com]
>>>>>> Sent: Friday, October 03, 2014 3:03 AM
>>>>>> To:dev@oodt.apache.org
>>>>>> Subject: Re: how to use MetadataBasedFileVersioner properly
>>>>>>
>>>>>> You?re almost there Val!
>>>>>>
>>>>>> Unfortunately Versioners right now aren?t configurable from product
>>>>>> type policy  (would be great to capture this in a JIRA issue, here:
>>>>>> https://issues.apache.org/jira/browse/OODT). If they were, it would
>>>>>> have picked up  your <property .. declaration of filePathSpec
below.
>>>>>> It?s been on my TODO list for a long time.
>>>>>>
>>>>>> Instead I created this:
>>>>>>
>>>>>> https://issues.apache.org/jira/browse/OODT-639
>>>>>>
>>>>>>
>>>>>> So you can amend your definition below (also note you always have
to
>>>>>> include  /[Filename] at the end to get the filename you want).
>>>>>>
>>>>>>     <type id="urn:oodt:EngineeringFile" name="EngineeringFile">
>>>>>>       <repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
>>>>>>       <versioner
>>>>>> class=?org.apache.oodt.cas.filemgr.versioning.ProductTypeMetVersioner?/>
>>>>>>       <description>The default product type for any kind of
>>>>>> file.</description>
>>>>>>       <metExtractors>
>>>>>>         <extractor
>>>>>>
>>>>>> class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor">
>>>>>>           <configuration>
>>>>>>             <!-- you can optionally include the envReplace tag
to turn
>>>>>> on/off  environment var replacement -->
>>>>>>             <property name="nsAware" value="true" />
>>>>>>             <property name="elementNs" value="CAS" />
>>>>>>             <property name="elements"
>>>>>> value="ProductReceivedTime,ProductName,ProductId" />
>>>>>>           </configuration>
>>>>>>         </extractor>
>>>>>>       </metExtractors>
>>>>>>       <metadata>
>>>>>>    <keyval>
>>>>>>       <key>filePathSpec</key>
>>>>>>       <val>/[YearDir]/[DoyDir]/[Filename]</val>
>>>>>>     </keyval>
>>>>>>
>>>>>> </metadata>
>>>>>>     </type>
>>>>>>
>>>>>>
>>>>>> See if that fixes it!
>>>>>>
>>>>>> Cheers,
>>>>>> Chris
>>>>>>
>>>>>> ------------------------
>>>>>> Chris Mattmann
>>>>>> chris.mattmann@gmail.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: "Mallder, Valerie"<Valerie.Mallder@jhuapl.edu>
>>>>>> Reply-To:<dev@oodt.apache.org>
>>>>>> Date: Thursday, October 2, 2014 at 2:56 PM
>>>>>> To:"dev@oodt.apache.org"  <dev@oodt.apache.org>
>>>>>> Subject: how to use MetadataBasedFileVersioner properly
>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> I am trying to use the MetadataBasedFileVersioner to store my
files
>>>>>>> in a subdirectory of the data/archive folder based on the values
of
>>>>>>> two metadata elements. Has anyone does this before, and if so,
can
>>>>>>> you give me some hints.
>>>>>>>
>>>>>>> My files need to be organized in subdirectories by year and day
of
>>>>>> year.
>>>>>>> So, lets say the file name is 'myfile', the year is 2014, and
the
>>>>>>> doy of year is 002. Then, the end result that I am looking for
is to
>>>>>>> have the final location of my file be:
>>>>>> 'data/archive/ops/eng/2014/002/myfile.
>>>>>>> However, in my log file, the INFO messages indicate that the
>>>>>>> generated final location reference is 'data/archive/ops/eng/myfile'.
>>>>>>> And it doesn't include the year and day of year at all. And the
>>>>>>> incorrect location leads to other errors. So I want to solve
this
>>>>>>> one first. Has anyone tried to do something like this before?
>>>>>>>
>>>>>>> Here is what I have done so far:
>>>>>>>
>>>>>>> Added two new elements to the .met file 'YearDir' and 'DoyDir'.
>>>>>>> Added these new elements to the elements.xml file.
>>>>>>> Added these new elements to the product in the
>>>>>>> product-type-element-map.xml file.
>>>>>>> And, I have made changes to the product-types.xml file, but here's
>>>>>>> where I am not sure I've done this properly.
>>>>>>>
>>>>>>> Here's what product type definition looks like:
>>>>>>>    <type id="urn:oodt:EngineeringFile" name="EngineeringFile">
>>>>>>>      <repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
>>>>>>>      <versioner
>>>>>>> class="org.apache.oodt.cas.filemgr.versioning.MetadataBasedFileVersio
>>>>>>> ner
>>>>>>> ">
>>>>>>>            <property name="filePathSpec" value="/[YearDir]/[DoyDir]/"
/>
>>>>>>>      </versioner>
>>>>>>>      <description>The default product type for any kind
of
>>>>>>> file.</description>
>>>>>>>      <metExtractors>
>>>>>>>        <extractor
>>>>>>> class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor"
>>>>>>>          <configuration>
>>>>>>>            <!-- you can optionally include the envReplace
tag to turn
>>>>>>> on/off environment var replacement -->
>>>>>>>            <property name="nsAware" value="true" />
>>>>>>>            <property name="elementNs" value="CAS" />
>>>>>>>            <property name="elements"
>>>>>>> value="ProductReceivedTime,ProductName,ProductId" />
>>>>>>>          </configuration>
>>>>>>>        </extractor>
>>>>>>>      </metExtractors>
>>>>>>>      <metadata/>
>>>>>>>    </type>
>>>>>>>
>>>>>>> Thanks in advance for any help or ideas you might have!
>>>>>>>
>>>>>>> Valerie
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Valerie A. Mallder
>>>>>>>
>>>>>>> New Horizons Deputy Mission System Engineer The Johns Hopkins
>>>>>>> University/Applied Physics Laboratory
>>>>>>> 11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
>>>>>>> 240-228-7846 (Office) 410-504-2233 (Blackberry)
>>>>>>>
>> --
>> *Tom Barber* | Technical Director
>>
>> meteorite bi
>> *T:* +44 20 8133 3730
>> *W:*www.meteorite.bi<http://www.meteorite.bi>  | *Skype:*  meteorite.consulting
>> *A:* Surrey Technology Centre, Surrey Research Park, Guildford, GU2 7YG, UK
>>
>
>
> -- 
> *Tom Barber* | Technical Director
>
> meteorite bi
> *T:* +44 20 8133 3730
> *W:* www.meteorite.bi | *Skype:* meteorite.consulting
> *A:* Surrey Technology Centre, Surrey Research Park, Guildford, GU2 
> 7YG, UK


-- 
*Tom Barber* | Technical Director

meteorite bi
*T:* +44 20 8133 3730
*W:* www.meteorite.bi | *Skype:* meteorite.consulting
*A:* Surrey Technology Centre, Surrey Research Park, Guildford, GU2 7YG, UK

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message