hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adamos Loizou (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4966) Put/Delete values cannot be tested with MRUnit
Date Wed, 06 Mar 2013 11:22:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13594621#comment-13594621

Adamos Loizou commented on HBASE-4966:

Hello, I've also experienced this issue with the latest 0.94.5 version. As Philip pointed
out, MRUnit assumes that equals(), hashCode() are overridden in order to validate the actual
output against the expected. Perhaps it could be considered an MRUnit shortcoming, not allowing
to intercept and custom-match the actual output. I have also worked around the problem with
a test-only implementation of Put that enforces the contract and use that implementation for
tests and the default for production. We do quite a bit of map/reduce with HBase on my team
so it would be really helpful to get this fixed. I've seen this issue with Hadoop MapWritable
and the convention there seems to be to try and meet the equals() hashCode() contract.
> Put/Delete values cannot be tested with MRUnit
> ----------------------------------------------
>                 Key: HBASE-4966
>                 URL: https://issues.apache.org/jira/browse/HBASE-4966
>             Project: HBase
>          Issue Type: Bug
>          Components: Client, mapreduce
>    Affects Versions: 0.90.4
>            Reporter: Nicholas Telford
>            Assignee: Nicholas Telford
>            Priority: Minor
> When using the IdentityTableReducer, which expects input values of either a Put or Delete
object, testing with MRUnit the Mapper with MRUnit is not possible because neither Put nor
Delete implement equals().
> We should implement equals() on both such that equality means:
> * Both objects are of the same class (in this case, Put or Delete)
> * Both objects are for the same key.
> * Both objects contain an equal set of KeyValues (applicable only to Put)
> KeyValue.equals() appears to already be implemented, but only checks for equality of
row key, column family and column qualifier - two KeyValues can be considered "equal" if they
contain different values. This won't work for testing.
> Instead, the Put.equals() and Delete.equals() implementations should do a "deep" equality
check on their KeyValues, like this:
> {code:java}
> myKv.equals(theirKv) && Bytes.equals(myKv.getValue(), theirKv.getValue());
> {code}
> NOTE: This would impact any code that relies on the existing "identity" implementation
of Put.equals() and Delete.equals(), therefore cannot be guaranteed to be backwards-compatible.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message