hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrzej Bialecki (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-120) Reading an ArrayWriter does not work because valueClass does not get initialized
Date Wed, 05 Apr 2006 07:13:46 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-120?page=comments#action_12373234 ] 

Andrzej Bialecki  commented on HADOOP-120:
------------------------------------------

Regarding the class name encoding: you could use the same trick that we use in Nutch, org.apache.nutch.crawl.MapWritable,
which uses a sort of dictionary encoding, and as long as you use only the "standard" types
the overhead is just 1 byte. For non-standard types, the type name is put into a dictionary
(once), and henceforth only 1 byte is used, too.

> Reading an ArrayWriter does not work because valueClass does not get initialized
> --------------------------------------------------------------------------------
>
>          Key: HADOOP-120
>          URL: http://issues.apache.org/jira/browse/HADOOP-120
>      Project: Hadoop
>         Type: Bug

>   Components: io
>  Environment: Red Hat  
>     Reporter: Dick King
>  Attachments: hadoop-120-fix.patch
>
> If you have a Reducer whose value type is an ArrayWriter it gets enstreamed alright but
at reconstruction type when ArrayWriter::readFields(DataInput in) runs on a DataInput that
has a nonempty ArrayWriter , newInstance fails trying to instantiate the null class.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message