oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Mattmann <chris.mattm...@gmail.com>
Subject Re: how to use MetadataBasedFileVersioner properly
Date Tue, 14 Oct 2014 19:32:47 GMT
No problem at all, Val. Keep up all the great work.

You will be there soon and we will be so happy when
you get there!

In the meanwhile we┬╣re here to help.

Cheers,
Chris

------------------------
Chris Mattmann
chris.mattmann@gmail.com




-----Original Message-----
From: "Mallder, Valerie" <Valerie.Mallder@jhuapl.edu>
Reply-To: <dev@oodt.apache.org>
Date: Tuesday, October 14, 2014 at 9:51 AM
To: "dev@oodt.apache.org" <dev@oodt.apache.org>
Subject: RE: how to use MetadataBasedFileVersioner properly

>Hi Chris,
>
>I'm taking a few minutes here to respond to all the folks who have
>offered me some help in the past couple weeks, and I don't know if I ever
>responded to this one from you.  Your help below was golden and it helped
>me make more progress at that time.  Thanks so much!
>
>Val
>
>
>
>Valerie A. Mallder
>New Horizons Deputy Mission System Engineer
>Johns Hopkins University/Applied Physics Laboratory
>
>> -----Original Message-----
>> From: Chris Mattmann [mailto:chris.mattmann@gmail.com]
>> Sent: Friday, October 03, 2014 3:03 AM
>> To: dev@oodt.apache.org
>> Subject: Re: how to use MetadataBasedFileVersioner properly
>>
>> You?re almost there Val!
>>
>> Unfortunately Versioners right now aren?t configurable from product
>>type policy
>> (would be great to capture this in a JIRA issue, here:
>> https://issues.apache.org/jira/browse/OODT). If they were, it would
>>have picked up
>> your <property .. declaration of filePathSpec below.
>> It?s been on my TODO list for a long time.
>>
>> Instead I created this:
>>
>> https://issues.apache.org/jira/browse/OODT-639
>>
>>
>> So you can amend your definition below (also note you always have to
>>include
>> /[Filename] at the end to get the filename you want).
>>
>>   <type id="urn:oodt:EngineeringFile" name="EngineeringFile">
>>     <repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
>>     <versioner
>> class=?org.apache.oodt.cas.filemgr.versioning.ProductTypeMetVersioner?/>
>>     <description>The default product type for any kind of
>>file.</description>
>>     <metExtractors>
>>       <extractor
>> 
>>class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor">
>>         <configuration>
>>           <!-- you can optionally include the envReplace tag to turn
>>on/off
>> environment var replacement -->
>>           <property name="nsAware" value="true" />
>>           <property name="elementNs" value="CAS" />
>>           <property name="elements"
>> value="ProductReceivedTime,ProductName,ProductId" />
>>         </configuration>
>>       </extractor>
>>     </metExtractors>
>>     <metadata>
>>  <keyval>
>>     <key>filePathSpec</key>
>>     <val>/[YearDir]/[DoyDir]/[Filename]</val>
>>   </keyval>
>>
>> </metadata>
>>   </type>
>>
>>
>> See if that fixes it!
>>
>> Cheers,
>> Chris
>>
>> ------------------------
>> Chris Mattmann
>> chris.mattmann@gmail.com
>>
>>
>>
>>
>> -----Original Message-----
>> From: "Mallder, Valerie" <Valerie.Mallder@jhuapl.edu>
>> Reply-To: <dev@oodt.apache.org>
>> Date: Thursday, October 2, 2014 at 2:56 PM
>> To: "dev@oodt.apache.org" <dev@oodt.apache.org>
>> Subject: how to use MetadataBasedFileVersioner properly
>>
>> >Hi All,
>> >
>> >I am trying to use the MetadataBasedFileVersioner to store my files in
>> >a subdirectory of the data/archive folder based on the values of two
>> >metadata elements. Has anyone does this before, and if so, can you give
>> >me some hints.
>> >
>> >My files need to be organized in subdirectories by year and day of
>>year.
>> >So, lets say the file name is 'myfile', the year is 2014, and the doy
>> >of year is 002. Then, the end result that I am looking for is to have
>> >the final location of my file be:
>>'data/archive/ops/eng/2014/002/myfile.
>> >
>> >However, in my log file, the INFO messages indicate that the generated
>> >final location reference is 'data/archive/ops/eng/myfile'.  And it
>> >doesn't include the year and day of year at all. And the incorrect
>> >location leads to other errors. So I want to solve this one first. Has
>> >anyone tried to do something like this before?
>> >
>> >Here is what I have done so far:
>> >
>> >Added two new elements to the .met file 'YearDir' and 'DoyDir'.
>> >Added these new elements to the elements.xml file.
>> >Added these new elements to the product in the
>> >product-type-element-map.xml file.
>> >And, I have made changes to the product-types.xml file, but here's
>> >where I am not sure I've done this properly.
>> >
>> >Here's what product type definition looks like:
>> >  <type id="urn:oodt:EngineeringFile" name="EngineeringFile">
>> >    <repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
>> >    <versioner
>> 
>>>class="org.apache.oodt.cas.filemgr.versioning.MetadataBasedFileVersioner
>>>">
>> >          <property name="filePathSpec" value="/[YearDir]/[DoyDir]/" />
>> >    </versioner>
>> >    <description>The default product type for any kind of
>> >file.</description>
>> >    <metExtractors>
>> >      <extractor
>> 
>>>class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor"
>>>>
>> >        <configuration>
>> >          <!-- you can optionally include the envReplace tag to turn
>> >on/off environment var replacement -->
>> >          <property name="nsAware" value="true" />
>> >          <property name="elementNs" value="CAS" />
>> >          <property name="elements"
>> >value="ProductReceivedTime,ProductName,ProductId" />
>> >        </configuration>
>> >      </extractor>
>> >    </metExtractors>
>> >    <metadata/>
>> >  </type>
>> >
>> >Thanks in advance for any help or ideas you might have!
>> >
>> >Valerie
>> >
>> >
>> >
>> >Valerie A. Mallder
>> >
>> >New Horizons Deputy Mission System Engineer The Johns Hopkins
>> >University/Applied Physics Laboratory
>> >11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
>> >240-228-7846 (Office) 410-504-2233 (Blackberry)
>> >
>>
>



Mime
View raw message