hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-742) Implement of Hama RPC
Date Thu, 18 Jul 2013 08:54:48 GMT

    [ https://issues.apache.org/jira/browse/HAMA-742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13712140#comment-13712140

Edward J. Yoon commented on HAMA-742:

Avro was used as a default IO serialization protocol[1].

java.lang.ClassNotFoundException: Class org.apache.avro.io.DatumWriter not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1486)
	at org.apache.hadoop.io.serializer.SerializationFactory.add(SerializationFactory.java:70)
	at org.apache.hadoop.io.serializer.SerializationFactory.<init>(SerializationFactory.java:63)
	at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1173)
	at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:1103)
	at org.apache.hama.bsp.SequenceFileRecordWriter.<init>(SequenceFileRecordWriter.java:39)

1. http://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/serializer/SerializationFactory.java

So, we need to add avro dependency to lib, or "io.serializations" property to hama-default.xml.


I prefer the latter (set "io.serializations" to WritableSerialization).

{quote}why are DistributedCache.add/setLocalFiles(conf, files.toString()); commented out ?{quote}

Oh thanks, my fault! I have to fix this.
> Implement of Hama RPC 
> ----------------------
>                 Key: HAMA-742
>                 URL: https://issues.apache.org/jira/browse/HAMA-742
>             Project: Hama
>          Issue Type: Sub-task
>            Reporter: Edward J. Yoon
>            Assignee: Edward J. Yoon
>             Fix For: 0.6.3
>         Attachments: HAMA-742_v01.patch, HAMA-742_v02.patch
> To solve HDFS 2.0 compatibility issue, we have to change a lot of codes for Hadoop 2.0
RPC, moreover, yarn RPC doesn't support asynchronous call directly.
> Ultimately, we can pursue the performance and integrate more easily with hadoop multi-versions
by having our own RPC.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message