crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hansen,Chuck" <Chuck.Han...@Cerner.com>
Subject Writing MapFile through Crunch, issue reading through Hadoop
Date Mon, 09 Sep 2013 17:28:06 GMT
With Crunch versions prior to 0.7.x, there does not appear to be an _SUCCESS file written upon
completion, starting with 0.7.x there is.  This file (and any others not intended to be read
through [1]) appears to cause issue with [1].  This means writing a MapFile with crunch and
reading back with [1] works prior to 0.7.x, but starting with 0.7.x, [1] will throw an exception.

Is this a bug with Crunch and/or Hadoop?

[1] org.apache.hadoop.mapreduce.lib.output.MapFileOutputFormat.getReaders

Hadoop CDH versions used:

    <hadoopCoreVersion>2.0.0-mr1-cdh4.2.1</hadoopCoreVersion>

    <hadoop_commonAndHDFSVersion>2.0.0-cdh4.2.1</hadoop_commonAndHDFSVersion>

--
Chuck Hansen
Software Engineer, Record Dev
chuck.hansen@cerner.com<mailto:chuck.hansen@cerner.com> | 816-201-9629
Cerner Corporation | www.cerner.com<http://www.cerner.com/>

CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation
and are intended only for the addressee. The information contained in this message is confidential
and may constitute inside or non-public information under international, federal, or state
securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such
information is strictly prohibited and may be unlawful. If you are not the addressee, please
promptly delete this message and notify the sender of the delivery error by e-mail or you
may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024.

Mime
View raw message