hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-6949) Reduces RPC packet size for primitive arrays, especially long[], which is used at block reporting
Date Mon, 21 Mar 2011 21:16:11 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Suresh Srinivas updated HADOOP-6949:
------------------------------------

      Resolution: Fixed
    Release Note: 
Increments the RPC protocol version in org.apache.hadoop.ipc.Server from 4 to 5.
Introduces ArrayPrimitiveWritable for a much more efficient wire format to transmit arrays
of primitives over RPC. ObjectWritable uses the new writable for array of primitives for RPC
and continues to use existing format for on-disk data.

  was:
Increments the RPC protocol version in org.apache.hadoop.ipc.Server from 4 to 5.
Introduces a much more efficient wire format to transmit arrays of primitives.  This is done
in a way that is 100% backward compatible with previously persisted data, so any stored data
will still be readable.  However, data written in the new version cannot be read by a process
using the old version of protocol.  Only RPC uses the new format; file writing protocols for
application data are unchanged..

    Hadoop Flags: [Incompatible change, Reviewed]  (was: [Incompatible change])
          Status: Resolved  (was: Patch Available)

I committed the patch. Thank you Matt.

> Reduces RPC packet size for primitive arrays, especially long[], which is used at block
reporting
> -------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-6949
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6949
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io
>    Affects Versions: 0.22.0
>            Reporter: Navis
>            Assignee: Matt Foley
>             Fix For: 0.23.0
>
>         Attachments: ObjectWritable.diff, arrayprim.patch, arrayprim_v4.patch, arrayprim_v5.patch,
arrayprim_v6.patch, arrayprim_v7.patch
>
>   Original Estimate: 10m
>  Remaining Estimate: 10m
>
> Current implementation of oah.io.ObjectWritable marshals primitive array types as general
object array ; array type string + array length + (element type string + value)*n
> It would not be needed to specify each element types for primitive arrays.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message