crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabriel Reid (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-442) Trying to read specific avros, getting ClassCastException
Date Tue, 08 Jul 2014 19:15:05 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14055335#comment-14055335
] 

Gabriel Reid commented on CRUNCH-442:
-------------------------------------

Ok, that would be great if you could post if that works or not.

The reason the snippet of code can't just be taken and put into a unit test is that the whole
issue revolves around how the hadoop command itself sets up its class loaders. There are actually
quite a few places in the existing integration tests where Avro specific records are read
via materialize, and they all work, because all of the Avro specific classes are on the root
class path. When a job is run with the hadoop command, the Avro specific classes are loaded
from the job jar using (I think) the context class loader.

> Trying to read specific avros, getting ClassCastException
> ---------------------------------------------------------
>
>                 Key: CRUNCH-442
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-442
>             Project: Crunch
>          Issue Type: Bug
>          Components: IO
>    Affects Versions: 0.10.0
>            Reporter: Jason Gauci
>
> I have code that looks like this:
> 		ArrayList<MySpecificAvro> data = Lists.newArrayList(pipeline.read(From.avroFile(
> 				"/user/jgauci/data.avro",
> 				Avros.specifics(MySpecificAvro.class))).materialize());
> 		for (MySpecificAvro msa : data) {
> 			System.out.println(msa.getName());
> 		}
> And I get this error:
> Exception in thread "main" java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record
cannot be cast to MySpecificAvro
> The error fires inside the for loop after the materialize has completed.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message