oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantinos Mavrommatis <kmavromma...@celgene.com>
Subject Re: How to process files in a sorted order
Date Sat, 16 Nov 2013 01:14:29 GMT
Thanks Cameron,
I will try to play with it and I will post a solution if I find one.
Konstantinos

On 11/13/13 3:18 PM, "Cameron Goodale" <sigep311@gmail.com> wrote:

>Konstantinos,
>
>My name is Cameron and I am a committer on the Apache OODT project.  I am
>not familiar with the internals of crawler, but I don't believe there is a
>way to accomplish your goal of enforcing a sorting algorithm within the
>crawler config.  I think you will have to write your own crawler that will
>implement your sorting logic.
>
>
>
>Sincerely,
>
>
>Cameron Goodale
>
>
>On Thu, Nov 7, 2013 at 7:44 PM, Konstantinos Mavrommatis <
>kmavrommatis@celgene.com> wrote:
>
>> Hi,
>> In my environment I am using cas-crawler to process directories of 1000s
>> of files. The metadata for these files are extracted automatically using
>> the mimetypes definitions and small wrapper scripts.
>> In these directories some of the files are derived from other files and
>> metadata from the older files need to be transferred to the newer file.
>> In order to achieve this I need to have the files processed by the
>> cas-crawler starting from the older file to the newer file or in other
>> cases in alphabetical order..
>> Any ideas how this can be achieved?
>>
>> The crawler command I currently use is:
>> ./crawler_launcher --operation --launchAutoCrawler --productPath
>>$FILEPATH
>> --filemgrUrl $FMURL --clientTransferer
>> org.apache.oodt.cas.filemgr.datatransfer.InPlaceDataTransferFactory
>>  --mimeExtr
>> actorRepo ../policy/mime-extractor-map.xml
>>
>> Thanks in advance for your help
>> Konstantinos
>>
>> *********************************************************
>> THIS ELECTRONIC MAIL MESSAGE AND ANY ATTACHMENT IS
>> CONFIDENTIAL AND MAY CONTAIN LEGALLY PRIVILEGED
>> INFORMATION INTENDED ONLY FOR THE USE OF THE INDIVIDUAL
>> OR INDIVIDUALS NAMED ABOVE.
>> If the reader is not the intended recipient, or the
>> employee or agent responsible to deliver it to the
>> intended recipient, you are hereby notified that any
>> dissemination, distribution or copying of this
>> communication is strictly prohibited. If you have
>> received this communication in error, please reply to the
>> sender to notify us of the error and delete the original
>> message. Thank You.
>> *********************************************************
>>
>>
>
>
>-- 
>
>Sent from a Tin Can attached to a String

*********************************************************
THIS ELECTRONIC MAIL MESSAGE AND ANY ATTACHMENT IS
CONFIDENTIAL AND MAY CONTAIN LEGALLY PRIVILEGED
INFORMATION INTENDED ONLY FOR THE USE OF THE INDIVIDUAL
OR INDIVIDUALS NAMED ABOVE. 
If the reader is not the intended recipient, or the
employee or agent responsible to deliver it to the
intended recipient, you are hereby notified that any
dissemination, distribution or copying of this
communication is strictly prohibited. If you have
received this communication in error, please reply to the
sender to notify us of the error and delete the original
message. Thank You.
********************************************************* 


Mime
View raw message