hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David S. Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6009) Changes for HBASE-5209 are technically incompatible
Date Thu, 17 May 2012 13:53:08 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13277810#comment-13277810

David S. Wang commented on HBASE-6009:

I looked at the total size field option for this, starting from the write case.

To calculate total size written, you have to know how many bytes were written for each write()
call on ClusterStatus, including any objects contained inside it.

The DataOutput interface for Writables doesn't have a way to return how many bytes were written
to the stream.  This is not a problem for primitive types as we can figure that out trivially.
 Even for somewhat more complicated situations such as modified UTF-8s written with the writeUTF
call, the number of written bytes for a String can at least be calculated based on the formula
for modified UTF-8 conversion.

However, for calls to Object's write functions (e.g. for HRegionLoad), this becomes somewhat
more problematic as there is no obvious answer as to how many bytes were written.  We could
use reflection to grab the fields, but then there is no guarantee that all of the fields of
the Object are actually written to the stream when write() is called.  So you'd have to introduce
some hardcoded way of knowing how much was written for each Object, which is Bad.

I'm tempted to say that we shouldn't add any more fields to ClusterStatus or similar APIs
until 0.96, when hopefully our wire compatibility efforts will kick in and we can do this
in a compatible way without having to jump through hoops.
> Changes for HBASE-5209 are technically incompatible
> ---------------------------------------------------
>                 Key: HBASE-6009
>                 URL: https://issues.apache.org/jira/browse/HBASE-6009
>             Project: HBase
>          Issue Type: Bug
>          Components: master
>    Affects Versions: 0.92.1, 0.94.0
>            Reporter: David S. Wang
> The additions to add backup masters to ClusterStatus are technically incompatible between
clients and servers.  Older clients will basically not read the extra bits that the newer
server pushes for the backup masters, thus screwing up the serialization for the next blob
in the pipe.
> For the Writable, we can add a total size field for ClusterStatus at the beginning, or
we can have start and end markers.  I can make a patch for either approach; interested in
whatever folks have to suggest.  Would be good to get this in soon to limit the damage to
0.92.1 (don't know if we can get this in in time for 0.94.0).
> Either change will make us forward-compatible starting with when the change goes in,
but will not fix the backwards incompatibility, which we will have to mark with a release
note as there have already been releases with this change.
> Hopefully we can do this in a cleaner way when wire compat rolls around in 0.96.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message