hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Milind Bhandarkar (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-941) Enhancements to Hadoop record I/O - Part 1
Date Wed, 28 Feb 2007 21:53:50 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Milind Bhandarkar updated HADOOP-941:
-------------------------------------

    Attachment: jute-patch.txt

Diffs between this patch and previous one:

The contentious issue of alllowing the record i/o translator to have an option to generate
code that could be used outside the Hadoop context (for which no consensus could be reached
so far) has been separated into a different issue, and those changes have been removed. i.e.
generated records always implement WritableComparable and provide a raw comparator.

David Bowen pointed out that Binary deserialization for longs did not handle negative numbers.
This has been fixed, and test added. This bug existed in c++ deserialization as welll, and
has been fixed.

Every generated record is now cloneable. i.e. it implements Object.clone(). Test added.

Buffer class now has a toString() method that spits out its string representation as sequence
of hex-pairs. In addition it has a toString(String charsetName) for application that want
to use buffer it to store non-utf8 string.


> Enhancements to Hadoop record I/O - Part 1
> ------------------------------------------
>
>                 Key: HADOOP-941
>                 URL: https://issues.apache.org/jira/browse/HADOOP-941
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: record
>    Affects Versions: 0.10.1
>         Environment: All
>            Reporter: Milind Bhandarkar
>         Assigned To: Milind Bhandarkar
>         Attachments: jute-patch.txt
>
>
> Hadoop record I/O can be used effectively outside of Hadoop. It would increase its utility
if developers can use it without having to import hadoop classes, or having to depend on Hadoop
jars. Following changes to the current translator and runtime are proposed.
> Proposed Changes:
> 1. Use java.lang.String as a native type for ustring (instead of Text.)
> 2. Provide a Buffer class as a native Java type for buffer (instead of BytesWritable),
so that later BytesWritable could be implemented as following DDL:
> module org.apache.hadoop.io {
>   record BytesWritable {
>     buffer value;
>   }
> }
> 3. Member names in generated classes should not have prefixes 'm' before their names.
In the above example, the private member name would be 'value' not 'mvalue' as it is done
now.
> 4. Convert getters and setters to have CamelCase. e.g. in the above example the getter
will be:
>   public Buffer getValue();
> 5. Generate clone() methods for records in Java i.e. the generated classes should implement
Cloneable.
> 6. Make generated Java codes for maps and vectors use Java generics.
> These are the proposed user-visible changes. Internally, the translator will be restructured
so that it is easier to plug-in translators for different targets.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message