oodt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lewis John Mcgibbney <lewis.mcgibb...@gmail.com>
Subject Re: Transition from OODT 0.6 to 0.12 cannot find extractor specifications
Date Mon, 04 Apr 2016 22:24:18 GMT
Hi Konstantinos,
It appears to be happening with a tar.gz file as well right?

WARNING: No extractor specs specified for
/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/cas-crawler-04-02-16.log.gz

I wonder if it is the file names... However I would be extremely surprised
as I've seen some much more verbose file naming.
Lewis

On Saturday, April 2, 2016, Konstantinos Mavrommatis <
kmavrommatis@celgene.com> wrote:

> Hi,
> I am trying to replicate a fully functional service that I had setup long
> time ago using OODT 0.6 but I am having the following problem that does not
> allow me to ingest files. When I try to ingest files with the extension
> fastq.gz I get the line:
> WARNING: No extractor specs specified for
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/E837642_R1.fastq.gz
> Apr 02, 2016 10:12:14 PM org.apache.oodt.cas.crawl.ProductCrawler
> handleFile
> And of course the file is not ingested. This process works without problem
> with OODT 0.6 on a different server.
>
> The crawler command I am running is:
> ./crawler_launcher \
> --operation \
> --launchAutoCrawler \
> --productPath $FILEPATH \
> --filemgrUrl $OODT_FILEMGR_URL \
> --clientTransferer
> org.apache.oodt.cas.filemgr.datatransfer.InPlaceDataTransferFactory \
> --mimeExtractorRepo ../policy/mime-extractor-map.xml \
> --noRecur \
> --crawlForDirs 2>&1
>
>
>
> I have setup OODT 0.12 on a server which runs FM listening to port 9000.
> From a client machine I have verified that I can use FM to ingest products.
> I am now trying to use crawler to crawl and ingest all files in a
> directory. Since I have non standard MIME types in these directories I have
> done the following:
> 1. Added my own mime types in policy/mimetypes.xml eg
>   <mime-type type="text/fastq">
>                 <glob pattern="*.fastq"/>
>                 <glob pattern="*.fastq.gz"/>
>                 <glob pattern="*.fastq.bz"/>
>                 <glob pattern="*.fastq.bz2"/>
>                 <glob pattern="*.fastq.bzip"/>
>                 <glob pattern="*.fq"/>
>                 <glob pattern="*.fq.gz"/>
>                 <glob pattern="*.fq.bz"/>
>                 <glob pattern="*.fq.bz2"/>
>                 <glob pattern="*.fq.bzip"/>
>         </mime-type>
> 2. created the file policy/mime-extractor-map.xml
>
>         <mime type="text/fastq">
>                 <extractor
> class="org.apache.oodt.cas.metadata.extractors.ExternMetExtractor">
>                         <config
> file="/apache-oodt/crawler/bin/fastq.config"/>
>                         <preCondComparators>
>                                 <preCondComparator
> id="CheckThatDataFileSizeIsGreaterThanZero"/>
>                         </preCondComparators>
>                 </extractor>
>         </mime>
>
> 3. created the file fastq.config
> <?xml version="1.0" encoding="UTF-8"?>
> <cas:externextractor xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
>   <exec workingDir="">
>
> <extractorBinPath>/apache-oodt/crawler/bin/MetExtractorNGS.pl</extractorBinPath>
>       <args>
>          <arg isDataFile="true"></arg>
>         <arg>fastq</arg>
>       </args>
>    </exec>
> </cas:externextractor>
>
>
>
> The MetExtractorNGS.pl is a small perl script that opens the file to be
> ingested, gets some information and stores it in the .met file that
> corresponds to the file to be ingested and have manually verified that
> works as expected producing the correct met file.
>
> What am I missing here? Any ideas comments suggestions will be greatly
> appreciated.
> Thanks in advance for any help
> Kostas
>
>
>
> PS1 The full output from running the crawler command follows:
>
>
> Setting property 'StdProductCrawler.filemgrUrl'
> Setting property 'MetExtractorProductCrawler.filemgrUrl'
> Setting property 'AutoDetectProductCrawler.filemgrUrl'
> Setting property 'StdProductCrawler.clientTransferer'
> Setting property 'MetExtractorProductCrawler.clientTransferer'
> Setting property 'AutoDetectProductCrawler.clientTransferer'
> Setting property 'StdProductCrawler.noRecur'
> Setting property 'MetExtractorProductCrawler.noRecur'
> Setting property 'AutoDetectProductCrawler.noRecur'
> Setting property 'AutoDetectProductCrawler.mimeExtractorRepo'
> Setting property 'StdProductCrawler.productPath'
> Setting property 'MetExtractorProductCrawler.productPath'
> Setting property 'AutoDetectProductCrawler.productPath'
> Apr 02, 2016 10:12:13 PM
> org.springframework.beans.factory.config.PropertyOverrideConfigurer
> processKey
> FINE: Property 'AutoDetectProductCrawler.noRecur' set to value [true]
> Apr 02, 2016 10:12:13 PM
> org.springframework.beans.factory.config.PropertyOverrideConfigurer
> processKey
> FINE: Property 'StdProductCrawler.productPath' set to value
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq]
> Apr 02, 2016 10:12:13 PM
> org.springframework.beans.factory.config.PropertyOverrideConfigurer
> processKey
> FINE: Property 'MetExtractorProductCrawler.noRecur' set to value [true]
> Apr 02, 2016 10:12:13 PM
> org.springframework.beans.factory.config.PropertyOverrideConfigurer
> processKey
> FINE: Property 'AutoDetectProductCrawler.mimeExtractorRepo' set to value
> [../policy/mime-extractor-map.xml]
> Apr 02, 2016 10:12:13 PM
> org.springframework.beans.factory.config.PropertyOverrideConfigurer
> processKey
> FINE: Property 'MetExtractorProductCrawler.clientTransferer' set to value
> [org.apache.oodt.cas.filemgr.datatransfer.InPlaceDataTransferFactory]
> Apr 02, 2016 10:12:13 PM
> org.springframework.beans.factory.config.PropertyOverrideConfigurer
> processKey
> FINE: Property 'AutoDetectProductCrawler.filemgrUrl' set to value [
> http://192.168.8.44:9000]
> Apr 02, 2016 10:12:13 PM
> org.springframework.beans.factory.config.PropertyOverrideConfigurer
> processKey
> FINE: Property 'AutoDetectProductCrawler.clientTransferer' set to value
> [org.apache.oodt.cas.filemgr.datatransfer.InPlaceDataTransferFactory]
> Apr 02, 2016 10:12:13 PM
> org.springframework.beans.factory.config.PropertyOverrideConfigurer
> processKey
> FINE: Property 'StdProductCrawler.noRecur' set to value [true]
> Apr 02, 2016 10:12:13 PM
> org.springframework.beans.factory.config.PropertyOverrideConfigurer
> processKey
> FINE: Property 'StdProductCrawler.filemgrUrl' set to value [
> http://192.168.8.44:9000]
> Apr 02, 2016 10:12:13 PM
> org.springframework.beans.factory.config.PropertyOverrideConfigurer
> processKey
> FINE: Property 'AutoDetectProductCrawler.productPath' set to value
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq]
> Apr 02, 2016 10:12:13 PM
> org.springframework.beans.factory.config.PropertyOverrideConfigurer
> processKey
> FINE: Property 'MetExtractorProductCrawler.filemgrUrl' set to value [
> http://192.168.8.44:9000]
> Apr 02, 2016 10:12:13 PM
> org.springframework.beans.factory.config.PropertyOverrideConfigurer
> processKey
> FINE: Property 'StdProductCrawler.clientTransferer' set to value
> [org.apache.oodt.cas.filemgr.datatransfer.InPlaceDataTransferFactory]
> Apr 02, 2016 10:12:13 PM
> org.springframework.beans.factory.config.PropertyOverrideConfigurer
> processKey
> FINE: Property 'MetExtractorProductCrawler.productPath' set to value
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq]
> Apr 02, 2016 10:12:13 PM org.apache.oodt.cas.crawl.ProductCrawler crawl
> INFO: Crawling
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq
> Apr 02, 2016 10:12:13 PM org.apache.oodt.cas.crawl.ProductCrawler
> handleFile
> INFO: Handling file
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/E837642_R1.fastq.gz
> Apr 02, 2016 10:12:14 PM
> org.apache.oodt.cas.crawl.AutoDetectProductCrawler passesPreconditions
> WARNING: No extractor specs specified for
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/E837642_R1.fastq.gz
> Apr 02, 2016 10:12:14 PM org.apache.oodt.cas.crawl.ProductCrawler
> handleFile
> WARNING: Failed to pass preconditions for ingest of product:
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/E837642_R1.fastq.gz]
> Apr 02, 2016 10:12:14 PM org.apache.oodt.cas.crawl.ProductCrawler
> handleFile
> INFO: Handling file
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/E837642_R1.fastq.gz.met
> Apr 02, 2016 10:12:14 PM
> org.apache.oodt.cas.metadata.preconditions.PreCondEvalUtils eval
> INFO: Passed precondition comparator id
> CheckThatDataFileSizeIsGreaterThanZero
> Apr 02, 2016 10:12:14 PM
> org.apache.oodt.cas.metadata.extractors.ExternMetExtractor extrMetadata
> INFO: Generating met file for product file:
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/E837642_R1.fastq.gz.met]
> Apr 02, 2016 10:12:14 PM
> org.apache.oodt.cas.metadata.extractors.ExternMetExtractor extrMetadata
> INFO: Executing command line:
> [/celgene/software/apache-oodt/crawler/bin/MetExtractorNGS.pl
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/E837642_R1.fastq.gz.met
> text ] with workingDir:
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq]
> to extract metadata
> OUTPUT: [WARN : MetExtractorNGS - 2016/04/02 22:12:15] - Input file
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/E837642_R1.fastq.gz.met
> will be ignored. .met files are not processed !
> Apr 02, 2016 10:12:15 PM org.apache.oodt.cas.crawl.ProductCrawler
> handleFile
> SEVERE: Failed to get metadata for product : Met extractor failed to
> create metadata file
> org.apache.oodt.cas.metadata.exceptions.MetExtractionException: Met
> extractor failed to create metadata file
>         at
> org.apache.oodt.cas.metadata.extractors.ExternMetExtractor.extrMetadata(ExternMetExtractor.java:120)
>         at
> org.apache.oodt.cas.metadata.AbstractMetExtractor.extractMetadata(AbstractMetExtractor.java:74)
>         at
> org.apache.oodt.cas.crawl.AutoDetectProductCrawler.getMetadataForProduct(AutoDetectProductCrawler.java:84)
>         at
> org.apache.oodt.cas.crawl.ProductCrawler.handleFile(ProductCrawler.java:136)
>         at
> org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:104)
>         at
> org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:74)
>         at
> org.apache.oodt.cas.crawl.cli.action.CrawlerLauncherCliAction.execute(CrawlerLauncherCliAction.java:58)
>         at
> org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
>         at
> org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:188)
>         at
> org.apache.oodt.cas.crawl.CrawlerLauncher.main(CrawlerLauncher.java:36)
>
> Apr 02, 2016 10:12:15 PM org.apache.oodt.cas.crawl.ProductCrawler
> handleFile
> INFO: Handling file
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/E837642_R2.fastq.gz
> Apr 02, 2016 10:12:15 PM
> org.apache.oodt.cas.crawl.AutoDetectProductCrawler passesPreconditions
> WARNING: No extractor specs specified for
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/E837642_R2.fastq.gz
> Apr 02, 2016 10:12:15 PM org.apache.oodt.cas.crawl.ProductCrawler
> handleFile
> WARNING: Failed to pass preconditions for ingest of product:
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/E837642_R2.fastq.gz]
> Apr 02, 2016 10:12:15 PM org.apache.oodt.cas.crawl.ProductCrawler
> handleFile
> INFO: Handling file
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/E837642_R2.fastq.gz.met
> Apr 02, 2016 10:12:15 PM
> org.apache.oodt.cas.metadata.preconditions.PreCondEvalUtils eval
> INFO: Passed precondition comparator id
> CheckThatDataFileSizeIsGreaterThanZero
> Apr 02, 2016 10:12:16 PM
> org.apache.oodt.cas.metadata.extractors.ExternMetExtractor extrMetadata
> INFO: Generating met file for product file:
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/E837642_R2.fastq.gz.met]
> Apr 02, 2016 10:12:16 PM
> org.apache.oodt.cas.metadata.extractors.ExternMetExtractor extrMetadata
> INFO: Executing command line:
> [/celgene/software/apache-oodt/crawler/bin/MetExtractorNGS.pl
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/E837642_R2.fastq.gz.met
> text ] with workingDir:
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq]
> to extract metadata
> OUTPUT: [WARN : MetExtractorNGS - 2016/04/02 22:12:16] - Input file
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/E837642_R2.fastq.gz.met
> will be ignored. .met files are not processed !
> Apr 02, 2016 10:12:17 PM org.apache.oodt.cas.crawl.ProductCrawler
> handleFile
> SEVERE: Failed to get metadata for product : Met extractor failed to
> create metadata file
> org.apache.oodt.cas.metadata.exceptions.MetExtractionException: Met
> extractor failed to create metadata file
>         at
> org.apache.oodt.cas.metadata.extractors.ExternMetExtractor.extrMetadata(ExternMetExtractor.java:120)
>         at
> org.apache.oodt.cas.metadata.AbstractMetExtractor.extractMetadata(AbstractMetExtractor.java:74)
>         at
> org.apache.oodt.cas.crawl.AutoDetectProductCrawler.getMetadataForProduct(AutoDetectProductCrawler.java:84)
>         at
> org.apache.oodt.cas.crawl.ProductCrawler.handleFile(ProductCrawler.java:136)
>         at
> org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:104)
>         at
> org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:74)
>         at
> org.apache.oodt.cas.crawl.cli.action.CrawlerLauncherCliAction.execute(CrawlerLauncherCliAction.java:58)
>         at
> org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
>         at
> org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:188)
>         at
> org.apache.oodt.cas.crawl.CrawlerLauncher.main(CrawlerLauncher.java:36)
>
> Apr 02, 2016 10:12:17 PM org.apache.oodt.cas.crawl.ProductCrawler
> handleFile
> INFO: Handling file
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/cas-crawler-04-02-16.log.gz
> Apr 02, 2016 10:12:17 PM
> org.apache.oodt.cas.crawl.AutoDetectProductCrawler passesPreconditions
> WARNING: No extractor specs specified for
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/cas-crawler-04-02-16.log.gz
> Apr 02, 2016 10:12:17 PM org.apache.oodt.cas.crawl.ProductCrawler
> handleFile
> WARNING: Failed to pass preconditions for ingest of product:
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/cas-crawler-04-02-16.log.gz]
> Apr 02, 2016 10:12:17 PM org.apache.oodt.cas.crawl.ProductCrawler
> handleFile
> INFO: Handling file
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/cas-crawler-04-02-16.tar.gz
> Apr 02, 2016 10:12:17 PM
> org.apache.oodt.cas.crawl.AutoDetectProductCrawler passesPreconditions
> WARNING: No extractor specs specified for
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/cas-crawler-04-02-16.tar.gz
> Apr 02, 2016 10:12:17 PM org.apache.oodt.cas.crawl.ProductCrawler
> handleFile
> WARNING: Failed to pass preconditions for ingest of product:
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/cas-crawler-04-02-16.tar.gz]
> Apr 02, 2016 10:12:17 PM org.apache.oodt.cas.crawl.ProductCrawler
> handleFile
> INFO: Handling file
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/cas-crawler-mnt-celgene.rnd.combio.mmgp.external-TestSeqData-RNA-Seq-RawData-fastq-04-02-16.tar.gz
> Apr 02, 2016 10:12:17 PM
> org.apache.oodt.cas.crawl.AutoDetectProductCrawler passesPreconditions
> WARNING: No extractor specs specified for
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/cas-crawler-mnt-celgene.rnd.combio.mmgp.external-TestSeqData-RNA-Seq-RawData-fastq-04-02-16.tar.gz
> Apr 02, 2016 10:12:17 PM org.apache.oodt.cas.crawl.ProductCrawler
> handleFile
> WARNING: Failed to pass preconditions for ingest of product:
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/cas-crawler-mnt-celgene.rnd.combio.mmgp.external-TestSeqData-RNA-Seq-RawData-fastq-04-02-16.tar.gz]
> Apr 02, 2016 10:12:17 PM org.apache.oodt.cas.crawl.ProductCrawler
> handleFile
> INFO: Handling file
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/test
> Apr 02, 2016 10:12:17 PM
> org.apache.oodt.cas.metadata.preconditions.PreCondEvalUtils eval
> INFO: Passed precondition comparator id
> CheckThatDataFileSizeIsGreaterThanZero
> Apr 02, 2016 10:12:17 PM
> org.apache.oodt.cas.metadata.extractors.ExternMetExtractor extrMetadata
> INFO: Generating met file for product file:
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/test]
> Apr 02, 2016 10:12:17 PM
> org.apache.oodt.cas.metadata.extractors.ExternMetExtractor extrMetadata
> INFO: Executing command line:
> [/celgene/software/apache-oodt/crawler/bin/MetExtractorNGS.pl
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/test
> text ] with workingDir:
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq]
> to extract metadata
> OUTPUT: [DEBUG : MetExtractorNGS - 2016/04/02 22:12:18] - Accessing NGS
> server at http://192.168.8.44:8082/RPC2
> OUTPUT: [DEBUG : metadataPrepare - 2016/04/02 22:12:18] - addMetadata:
> metadata for file_host are not in array format.Converting..
> OUTPUT: [DEBUG : metadataPrepare - 2016/04/02 22:12:18] - addMetadata:
> adding key/value [file_host]/[ip-192-168-8-66]
> OUTPUT: [DEBUG : metadataPrepare - 2016/04/02 22:12:18] - addMetadata:
> metadata for ProductType are not in array format.Converting..
> OUTPUT: [DEBUG : metadataPrepare - 2016/04/02 22:12:18] - addMetadata:
> adding key/value [ProductType]/[GenericFile]
> OUTPUT: [DEBUG : metadataPrepare - 2016/04/02 22:12:18] - addMetadata:
> metadata for ingest_user are not in array format.Converting..
> OUTPUT: [DEBUG : metadataPrepare - 2016/04/02 22:12:18] - addMetadata:
> adding key/value [ingest_user]/[kmavrommatis]
> OUTPUT: [DEBUG : MetExtractorNGS - 2016/04/02 22:12:18] - The file path is
> ARRAY(0x22d3f48). It will be added under the FilePath metadata field
> OUTPUT: [DEBUG : metadataPrepare - 2016/04/02 22:12:18] - addMetadata:
> metadata for FilePath are not in array format.Converting..
> OUTPUT: [DEBUG : metadataPrepare - 2016/04/02 22:12:18] - addMetadata:
> adding key/value
> [FilePath]/[/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/test]
> OUTPUT: [DEBUG : MetExtractorNGS - 2016/04/02 22:12:18] - This file is of
> type text
> OUTPUT: [DEBUG : MetExtractorNGS - 2016/04/02 22:12:18] - Storing metadata
> in file
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/test.met
> OUTPUT: [DEBUG : metadataPrepare - 2016/04/02 22:12:18] - Changing
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/test
> to
> OUTPUT: [DEBUG : metadataPrepare - 2016/04/02 22:12:18] -
> /mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/test
> OUTPUT: [DEBUG : metadataPrepare - 2016/04/02 22:12:18] - Changing
> kmavrommatis to
> OUTPUT: [DEBUG : metadataPrepare - 2016/04/02 22:12:18] - kmavrommatis
> OUTPUT: [DEBUG : metadataPrepare - 2016/04/02 22:12:18] - Changing
> GenericFile to
> OUTPUT: [DEBUG : metadataPrepare - 2016/04/02 22:12:18] - GenericFile
> OUTPUT: [DEBUG : metadataPrepare - 2016/04/02 22:12:18] - Changing
> ip-192-168-8-66 to
> OUTPUT: [DEBUG : metadataPrepare - 2016/04/02 22:12:18] - ip-192-168-8-66
> OUTPUT: [DEBUG : MetExtractorNGS - 2016/04/02 22:12:19] - Process finished
> SUCCESSFULLY
> Apr 02, 2016 10:12:19 PM
> org.apache.oodt.cas.metadata.extractors.ExternMetExtractor extrMetadata
> INFO: Met extraction successful for product file:
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/test]
> Apr 02, 2016 10:12:19 PM org.apache.oodt.cas.crawl.ProductCrawler ingest
> INFO: ProductCrawler: Ready to ingest product:
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/test]:
> ProductType: [GenericFile]
> Apr 02, 2016 10:12:19 PM org.apache.oodt.cas.filemgr.ingest.StdIngester
> setFileManager
> INFO: StdIngester: connected to file manager: [http://192.168.8.44:9000]
> Apr 02, 2016 10:12:19 PM
> org.apache.oodt.cas.filemgr.datatransfer.InPlaceDataTransferer
> setFileManagerUrl
> INFO: In Place Data Transfer to: [http://192.168.8.44:9000] enabled
> Apr 02, 2016 10:12:19 PM org.apache.oodt.cas.filemgr.ingest.StdIngester
> ingest
> INFO: StdIngester: ingesting product: ProductName: [test]: ProductType:
> [GenericFile]: FileLocation:
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/]
> Apr 02, 2016 10:12:19 PM
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient ingestProduct
> FINEST: File Manager Client: clientTransfer enabled: transfering product
> [test]
> Apr 02, 2016 10:12:19 PM
> org.apache.oodt.cas.filemgr.versioning.VersioningUtils
> createBasicDataStoreRefsFlat
> FINE: VersioningUtils: Generated data store ref:
> file:/opt/oodt/data/archive/test/test from origRef:
> file:/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/test
> Apr 02, 2016 10:12:19 PM org.apache.oodt.cas.crawl.ProductCrawler ingest
> INFO: Successfully ingested product:
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/test]:
> product id: 4c8de2da-265a-48c4-8380-3f1103dfecfc
> Apr 02, 2016 10:12:19 PM org.apache.oodt.cas.crawl.ProductCrawler
> handleFile
> INFO: Successful ingest of product:
> [/mnt/celgene.rnd.combio.mmgp.external/TestSeqData/RNA-Seq/RawData/fastq/test]
>
>
> *********************************************************
> THIS ELECTRONIC MAIL MESSAGE AND ANY ATTACHMENT IS
> CONFIDENTIAL AND MAY CONTAIN LEGALLY PRIVILEGED
> INFORMATION INTENDED ONLY FOR THE USE OF THE INDIVIDUAL
> OR INDIVIDUALS NAMED ABOVE.
> If the reader is not the intended recipient, or the
> employee or agent responsible to deliver it to the
> intended recipient, you are hereby notified that any
> dissemination, distribution or copying of this
> communication is strictly prohibited. If you have
> received this communication in error, please reply to the
> sender to notify us of the error and delete the original
> message. Thank You.
>


-- 
*Lewis*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message