Return-Path: X-Original-To: apmail-crunch-dev-archive@www.apache.org Delivered-To: apmail-crunch-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0BA0C10AD4 for ; Thu, 6 Feb 2014 23:19:21 +0000 (UTC) Received: (qmail 47525 invoked by uid 500); 6 Feb 2014 23:19:20 -0000 Delivered-To: apmail-crunch-dev-archive@crunch.apache.org Received: (qmail 47490 invoked by uid 500); 6 Feb 2014 23:19:20 -0000 Mailing-List: contact dev-help@crunch.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@crunch.apache.org Delivered-To: mailing list dev@crunch.apache.org Received: (qmail 47479 invoked by uid 500); 6 Feb 2014 23:19:19 -0000 Delivered-To: apmail-incubator-crunch-dev@incubator.apache.org Received: (qmail 47473 invoked by uid 99); 6 Feb 2014 23:19:19 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 06 Feb 2014 23:19:19 +0000 Date: Thu, 6 Feb 2014 23:19:19 +0000 (UTC) From: "Gabriel Reid (JIRA)" To: crunch-dev@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CRUNCH-338) TupleDeepCopier throws java.lang.ClassCastException: java.util.ArrayList cannot be cast to org.apache.avro.generic.IndexedRecord MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CRUNCH-338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13893961#comment-13893961 ] Gabriel Reid commented on CRUNCH-338: ------------------------------------- Thanks for the pseudo-code. The issue is a bug in the Cogroup class -- calling parallelDo twice on the entityGrouper table was just triggering the deep copier, which was crashing due to the Cogroup bug. > TupleDeepCopier throws java.lang.ClassCastException: java.util.ArrayList cannot be cast to org.apache.avro.generic.IndexedRecord > -------------------------------------------------------------------------------------------------------------------------------- > > Key: CRUNCH-338 > URL: https://issues.apache.org/jira/browse/CRUNCH-338 > Project: Crunch > Issue Type: Bug > Components: Core > Affects Versions: 0.8.2 > Reporter: Laxmikanth Samudrala > Assignee: Josh Wills > Attachments: CRUNCH-338.patch, ClassCastExceptionInDeepCopierIT.java, stack-trace.log > > > when PTable using twice and performing parallelDo causing java.lang.ClassCastException; when the same PTable used once for parallelDo not causing the exception or turning PTable to PTable, Tuple3.Collect>> and using paired PTable twice for parallelDo not causing any exception. > Note : The root cause seem's expressing the items passed to the TupleN is collection or a single instance. Suprisingly when we are performing parallelDo operation once on PTable is working with no exceptions and when performing parallelDo twice seem's try to make use of TupleDeepCopier.deepCopy; which is triggering the exception. > Template of Code : > Failure case : > PTable entityData = .....; > entityData.parallelDo(.....); > entityData.parallelDo(.....); > Success Case : > PTable entityData = .....; > entityData.parallelDo(.....); > Another success case : > PTable, Tuple3.Collect>> entityData; > entityData.parallelDo(.....); > entityData.parallelDo(.....); > stack trace for reference : > org.apache.crunch.CrunchRuntimeException: Error while deep copying avro value [.........] > at org.apache.crunch.types.avro.AvroDeepCopier.deepCopy(AvroDeepCopier.java:195) > at org.apache.crunch.types.avro.AvroDeepCopier$AvroSpecificDeepCopier.deepCopy(AvroDeepCopier.java:83) > at org.apache.crunch.types.avro.AvroType.getDetachedValue(AvroType.java:217) > at org.apache.crunch.types.TupleDeepCopier.deepCopy(TupleDeepCopier.java:60) > at org.apache.crunch.types.TupleDeepCopier.deepCopy(TupleDeepCopier.java:32) > at org.apache.crunch.types.avro.AvroType.getDetachedValue(AvroType.java:217) > at org.apache.crunch.lib.PTables.getDetachedValue(PTables.java:191) > at org.apache.crunch.types.avro.AvroTableType.getDetachedValue(AvroTableType.java:149) > at org.apache.crunch.types.avro.AvroTableType.getDetachedValue(AvroTableType.java:36) > at org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:54) > at org.apache.crunch.MapFn.process(MapFn.java:34) > at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:99) > at org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:56) > at org.apache.crunch.MapFn.process(MapFn.java:34) > at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:99) > at org.apache.crunch.impl.mr.run.RTNode.processIterable(RTNode.java:114) > at org.apache.crunch.impl.mr.run.CrunchReducer.reduce(CrunchReducer.java:57) > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:164) > at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:610) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:444) > at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:447) -- This message was sent by Atlassian JIRA (v6.1.5#6160)