hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7183) WritableComparator.get should not cache comparator objects
Date Tue, 19 Apr 2011 15:32:05 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13021625#comment-13021625
] 

Owen O'Malley commented on HADOOP-7183:
---------------------------------------

The test needs to be removed from the fix, since it is only testing the implementation detail
of this workaround fix.

Tests need to reflect the correct semantics of the code, not implementation details that will
be removed when someone has time to do the right fix.

> WritableComparator.get should not cache comparator objects
> ----------------------------------------------------------
>
>                 Key: HADOOP-7183
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7183
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Tom White
>            Priority: Blocker
>             Fix For: 0.20.3, 0.21.1, 0.22.0
>
>         Attachments: HADOOP-7183.patch
>
>
> HADOOP-6881 modified WritableComparator.get such that the constructed WritableComparator
gets saved back into the static map. This is fine for stateless comparators, but some comparators
have per-instance state, and thus this becomes thread-unsafe and causes errors in the shuffle
where multiple threads are doing comparisons. An example of a Comparator with per-instance
state is WritableComparator itself.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message