hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Feng Jiang (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-839) CLONE -hadoop should provide a common way to wrap instances with different types into one type
Date Wed, 20 Dec 2006 06:32:22 GMT
     [ http://issues.apache.org/jira/browse/HADOOP-839?page=all ]

Feng Jiang updated HADOOP-839:

    Attachment: patch.839.r488924

This is reopened version of issue#689.

The org.apache.hadoop.io.GenericWritable.java (at -r488924) has a little bit problem, though
it works in the most cases. But the registered classes are in the relationship of Parents-child,
it doesn't work correctly.

For example, ClassType1 is super class of ClassType2, the an object that wanted to be wrapped
is ClassType2. But in the set(Writable obj) method, the ClassType1 may be treated as corresponding
Class incorrectly.

Meanwhile, the old GenericWritable uses the O(N) algorithm to look up the corresponding class,
so the performance is not so good especially when set() method is called frequently.

The updated GenericWritable fixed this bug, and improved the performance. It has detailed
javadoc and sample to show users how to use it.

> CLONE -hadoop should provide a common way to wrap instances with different types into
one type
> ----------------------------------------------------------------------------------------------
>                 Key: HADOOP-839
>                 URL: http://issues.apache.org/jira/browse/HADOOP-839
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: io
>         Environment: All environment
>            Reporter: Feng Jiang
>             Fix For: 0.9.0
>         Attachments: patch.839.r488924
> When two sequence files, which have same Key type but different Value types, are mapped
out to reduce, multiple Value types is not allowed. In this case, we need a way to wrap instances
with different types into one class type to reduce.
> In current code, ObjectWritable is a sole choice. but it costs too many space, because
the class declaration will be appended into output file as a string for every Key-value pair.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message