crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabriel Reid (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CRUNCH-442) Trying to read specific avros, getting ClassCastException
Date Sun, 13 Jul 2014 06:29:04 GMT

     [ https://issues.apache.org/jira/browse/CRUNCH-442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gabriel Reid updated CRUNCH-442:
--------------------------------

    Attachment: CRUNCH-442.patch

Here's the patch to fix the issue. It also resolves the same issue for reflection-based Avro
records.

For reference, this is only a problem if Avro is in the lib directory of Hadoop. The class
loader used for loading the Avro classes is (by default) also used for loading specific and
reflection classes, and Hadoop uses different class loaders for loading things from its lib
directory and from the job jar. If Avro is only present in the job jar, this problem doesn't
come up. However, I know that at least CDH bundles the Avro jars in the lib directory, so
this fix will be needed for a while at least.

> Trying to read specific avros, getting ClassCastException
> ---------------------------------------------------------
>
>                 Key: CRUNCH-442
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-442
>             Project: Crunch
>          Issue Type: Bug
>          Components: IO
>    Affects Versions: 0.10.0
>            Reporter: Jason Gauci
>         Attachments: CRUNCH-442.patch
>
>
> I have code that looks like this:
> 		ArrayList<MySpecificAvro> data = Lists.newArrayList(pipeline.read(From.avroFile(
> 				"/user/jgauci/data.avro",
> 				Avros.specifics(MySpecificAvro.class))).materialize());
> 		for (MySpecificAvro msa : data) {
> 			System.out.println(msa.getName());
> 		}
> And I get this error:
> Exception in thread "main" java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record
cannot be cast to MySpecificAvro
> The error fires inside the for loop after the materialize has completed.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message