oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Mattmann <chris.mattm...@gmail.com>
Subject Re: how to use MetadataBasedFileVersioner properly
Date Fri, 03 Oct 2014 07:02:41 GMT
You¹re almost there Val!

Unfortunately Versioners right now aren¹t configurable from product
type policy (would be great to capture this in a JIRA issue, here:
https://issues.apache.org/jira/browse/OODT). If they were, it would
have picked up your <property .. declaration of filePathSpec below.
It¹s been on my TODO list for a long time.

Instead I created this:

https://issues.apache.org/jira/browse/OODT-639


So you can amend your definition below (also note you always have
to include /[Filename] at the end to get the filename you want).

  <type id="urn:oodt:EngineeringFile" name="EngineeringFile">
    <repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
    <versioner 
class=³org.apache.oodt.cas.filemgr.versioning.ProductTypeMetVersioner²/>
    <description>The default product type for any kind of
file.</description>
    <metExtractors>
      <extractor 
class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor">
        <configuration>
          <!-- you can optionally include the envReplace tag to turn
on/off environment var replacement -->
          <property name="nsAware" value="true" />
          <property name="elementNs" value="CAS" />
          <property name="elements"
value="ProductReceivedTime,ProductName,ProductId" />
        </configuration>
      </extractor>
    </metExtractors>
    <metadata>
 <keyval> 
    <key>filePathSpec</key>
    <val>/[YearDir]/[DoyDir]/[Filename]</val>
  </keyval>

</metadata>
  </type>


See if that fixes it!

Cheers,
Chris

------------------------
Chris Mattmann
chris.mattmann@gmail.com




-----Original Message-----
From: "Mallder, Valerie" <Valerie.Mallder@jhuapl.edu>
Reply-To: <dev@oodt.apache.org>
Date: Thursday, October 2, 2014 at 2:56 PM
To: "dev@oodt.apache.org" <dev@oodt.apache.org>
Subject: how to use MetadataBasedFileVersioner properly

>Hi All,
>
>I am trying to use the MetadataBasedFileVersioner to store my files in a
>subdirectory of the data/archive folder based on the values of two
>metadata elements. Has anyone does this before, and if so, can you give
>me some hints.
>
>My files need to be organized in subdirectories by year and day of year.
>So, lets say the file name is 'myfile', the year is 2014, and the doy of
>year is 002. Then, the end result that I am looking for is to have the
>final location of my file be:  'data/archive/ops/eng/2014/002/myfile.
>
>However, in my log file, the INFO messages indicate that the generated
>final location reference is 'data/archive/ops/eng/myfile'.  And it
>doesn't include the year and day of year at all. And the incorrect
>location leads to other errors. So I want to solve this one first. Has
>anyone tried to do something like this before?
>
>Here is what I have done so far:
>
>Added two new elements to the .met file 'YearDir' and 'DoyDir'.
>Added these new elements to the elements.xml file.
>Added these new elements to the product in the
>product-type-element-map.xml file.
>And, I have made changes to the product-types.xml file, but here's where
>I am not sure I've done this properly.
>
>Here's what product type definition looks like:
>  <type id="urn:oodt:EngineeringFile" name="EngineeringFile">
>    <repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
>    <versioner 
>class="org.apache.oodt.cas.filemgr.versioning.MetadataBasedFileVersioner">
>          <property name="filePathSpec" value="/[YearDir]/[DoyDir]/" />
>    </versioner>
>    <description>The default product type for any kind of
>file.</description>
>    <metExtractors>
>      <extractor 
>class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor">
>        <configuration>
>          <!-- you can optionally include the envReplace tag to turn
>on/off environment var replacement -->
>          <property name="nsAware" value="true" />
>          <property name="elementNs" value="CAS" />
>          <property name="elements"
>value="ProductReceivedTime,ProductName,ProductId" />
>        </configuration>
>      </extractor>
>    </metExtractors>
>    <metadata/>
>  </type>
>
>Thanks in advance for any help or ideas you might have!
>
>Valerie
>
>
>
>Valerie A. Mallder
>
>New Horizons Deputy Mission System Engineer
>The Johns Hopkins University/Applied Physics Laboratory
>11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
>240-228-7846 (Office) 410-504-2233 (Blackberry)
>



Mime
View raw message