hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8406) CompressionCodecFactory.CODEC_PROVIDERS iteration is thread-unsafe
Date Thu, 17 May 2012 01:00:29 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13277303#comment-13277303

Todd Lipcon commented on HADOOP-8406:

In practice I'm seeing this cause an occasional failure of the first HDFS saveNamespace operation
if there are many storage directories configured, because all of the saver threads start and
call this function at about the same time.
> CompressionCodecFactory.CODEC_PROVIDERS iteration is thread-unsafe
> ------------------------------------------------------------------
>                 Key: HADOOP-8406
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8406
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: io
>    Affects Versions: 2.0.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hadoop-8406.txt
> CompressionCodecFactory defines CODEC_PROVIDERS as:
> {code}
>   private static final ServiceLoader<CompressionCodec> CODEC_PROVIDERS =
>     ServiceLoader.load(CompressionCodec.class);
> {code}
> but this is a lazy collection which is thread-unsafe to iterate. We either need to synchronize
when we iterate over it, or we need to materialize it during class-loading time by copying
to a non-lazy collection

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message