oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (3980)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: Success! RE: how to use MetadataBasedFileVersioner properly
Date Fri, 03 Oct 2014 17:07:15 GMT
Awesome Val! :)

I think you¹re running into this:

https://issues.apache.org/jira/browse/OODT-751


In the meanwhile, try the 0.6 OPSUI, which you can grab
from here:

http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/


Grab the WAR file and drop it into your favorite container.
Make sure you have all of these environment variables installed
*before* starting Tomcat or Jetty, etc.:

https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+OPSUI


(see steps at bottom and replace 0.5 with 0.6)

I¹m working on a fix for OODT-751, at which point RADIX will
be pretty buff. Next steps at that point:

1. Release 0.7 and then encourage folks to get started by using
the Vagrant build, e.g.,

git clone https://github.com/apache/oodt
cd vagrant/radix
vagrant up

2. Fix OODT-491 and remaining workflow manager issues for Wengine
3. Work on Streaming OODT API with AMP Stack (via M. Starch et al)
and release in 0.8.
4. Conquer and win.

Thanks!

Cheers,
Chris

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++






-----Original Message-----
From: <Mallder>, Valerie <Valerie.Mallder@jhuapl.edu>
Reply-To: "dev@oodt.apache.org" <dev@oodt.apache.org>
Date: Friday, October 3, 2014 at 9:53 AM
To: "dev@oodt.apache.org" <dev@oodt.apache.org>
Subject: Success!  RE: how to use MetadataBasedFileVersioner properly

>Hi Chris,
>
>Yes, that indeed fixed it!  Thanks so much!  I now have 18 engineering
>files ingested. Whoo hoo!!
>
>Ok, so now, I would like to see what the opsui has to say about my
>ingested files. And here comes probably a really stupid question.  I am
>using Firefox on a Redhat Linux box, and I'm not that familiar with
>Firefox and it's settings, so this might be a browser setting issue. When
>I startup the opsui and select "File Catalog Browse" it shows that I have
>18 EngineeringFiles. Then, when I select File Catalog Browse from the
>strip of options under the logo, I get a message saying the page has
>expired. And here's the link that shows up in the address bar:
>http://localhost:8080/opsui/?wicket:interface=:7:fmbrowser_link::ILinkList
>ener::
>
>And I get the page expired message for all of the options that I select.
>Any idea's on this one??
>
>In the meantime though, I will start playing with an action for post
>ingestion success that simply makes another copy of all these engineering
>files and puts them another folder for Level 0 files.
>
>Thanks,
>Val
>
>
>
>
>Valerie A. Mallder
>New Horizons Deputy Mission System Engineer
>Johns Hopkins University/Applied Physics Laboratory
>
>
>> -----Original Message-----
>> From: Chris Mattmann [mailto:chris.mattmann@gmail.com]
>> Sent: Friday, October 03, 2014 3:03 AM
>> To: dev@oodt.apache.org
>> Subject: Re: how to use MetadataBasedFileVersioner properly
>>
>> You?re almost there Val!
>>
>> Unfortunately Versioners right now aren?t configurable from product
>>type policy
>> (would be great to capture this in a JIRA issue, here:
>> https://issues.apache.org/jira/browse/OODT). If they were, it would
>>have picked up
>> your <property .. declaration of filePathSpec below.
>> It?s been on my TODO list for a long time.
>>
>> Instead I created this:
>>
>> https://issues.apache.org/jira/browse/OODT-639
>>
>>
>> So you can amend your definition below (also note you always have to
>>include
>> /[Filename] at the end to get the filename you want).
>>
>>   <type id="urn:oodt:EngineeringFile" name="EngineeringFile">
>>     <repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
>>     <versioner
>> class=?org.apache.oodt.cas.filemgr.versioning.ProductTypeMetVersioner?/>
>>     <description>The default product type for any kind of
>>file.</description>
>>     <metExtractors>
>>       <extractor
>> 
>>class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor">
>>         <configuration>
>>           <!-- you can optionally include the envReplace tag to turn
>>on/off
>> environment var replacement -->
>>           <property name="nsAware" value="true" />
>>           <property name="elementNs" value="CAS" />
>>           <property name="elements"
>> value="ProductReceivedTime,ProductName,ProductId" />
>>         </configuration>
>>       </extractor>
>>     </metExtractors>
>>     <metadata>
>>  <keyval>
>>     <key>filePathSpec</key>
>>     <val>/[YearDir]/[DoyDir]/[Filename]</val>
>>   </keyval>
>>
>> </metadata>
>>   </type>
>>
>>
>> See if that fixes it!
>>
>> Cheers,
>> Chris
>>
>> ------------------------
>> Chris Mattmann
>> chris.mattmann@gmail.com
>>
>>
>>
>>
>> -----Original Message-----
>> From: "Mallder, Valerie" <Valerie.Mallder@jhuapl.edu>
>> Reply-To: <dev@oodt.apache.org>
>> Date: Thursday, October 2, 2014 at 2:56 PM
>> To: "dev@oodt.apache.org" <dev@oodt.apache.org>
>> Subject: how to use MetadataBasedFileVersioner properly
>>
>> >Hi All,
>> >
>> >I am trying to use the MetadataBasedFileVersioner to store my files in
>> >a subdirectory of the data/archive folder based on the values of two
>> >metadata elements. Has anyone does this before, and if so, can you give
>> >me some hints.
>> >
>> >My files need to be organized in subdirectories by year and day of
>>year.
>> >So, lets say the file name is 'myfile', the year is 2014, and the doy
>> >of year is 002. Then, the end result that I am looking for is to have
>> >the final location of my file be:
>>'data/archive/ops/eng/2014/002/myfile.
>> >
>> >However, in my log file, the INFO messages indicate that the generated
>> >final location reference is 'data/archive/ops/eng/myfile'.  And it
>> >doesn't include the year and day of year at all. And the incorrect
>> >location leads to other errors. So I want to solve this one first. Has
>> >anyone tried to do something like this before?
>> >
>> >Here is what I have done so far:
>> >
>> >Added two new elements to the .met file 'YearDir' and 'DoyDir'.
>> >Added these new elements to the elements.xml file.
>> >Added these new elements to the product in the
>> >product-type-element-map.xml file.
>> >And, I have made changes to the product-types.xml file, but here's
>> >where I am not sure I've done this properly.
>> >
>> >Here's what product type definition looks like:
>> >  <type id="urn:oodt:EngineeringFile" name="EngineeringFile">
>> >    <repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
>> >    <versioner
>> 
>>>class="org.apache.oodt.cas.filemgr.versioning.MetadataBasedFileVersioner
>>>">
>> >          <property name="filePathSpec" value="/[YearDir]/[DoyDir]/" />
>> >    </versioner>
>> >    <description>The default product type for any kind of
>> >file.</description>
>> >    <metExtractors>
>> >      <extractor
>> 
>>>class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor"
>>>>
>> >        <configuration>
>> >          <!-- you can optionally include the envReplace tag to turn
>> >on/off environment var replacement -->
>> >          <property name="nsAware" value="true" />
>> >          <property name="elementNs" value="CAS" />
>> >          <property name="elements"
>> >value="ProductReceivedTime,ProductName,ProductId" />
>> >        </configuration>
>> >      </extractor>
>> >    </metExtractors>
>> >    <metadata/>
>> >  </type>
>> >
>> >Thanks in advance for any help or ideas you might have!
>> >
>> >Valerie
>> >
>> >
>> >
>> >Valerie A. Mallder
>> >
>> >New Horizons Deputy Mission System Engineer The Johns Hopkins
>> >University/Applied Physics Laboratory
>> >11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
>> >240-228-7846 (Office) 410-504-2233 (Blackberry)
>> >
>>
>


Mime
View raw message