hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oleksandr Shevchenko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15865) ConcurrentModificationException in Configuration.overlay() method
Date Mon, 29 Oct 2018 16:25:01 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16667410#comment-16667410
] 

Oleksandr Shevchenko commented on HADOOP-15865:
-----------------------------------------------

Could someone review the attached changes?

Thanks.

> ConcurrentModificationException in Configuration.overlay() method
> -----------------------------------------------------------------
>
>                 Key: HADOOP-15865
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15865
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Oleksandr Shevchenko
>            Assignee: Oleksandr Shevchenko
>            Priority: Major
>         Attachments: HADOOP-15865.001.patch
>
>
> Configuration.overlay() is not thread-safe and can be the cause of ConcurrentModificationException
since we use iteration over Properties object. 
> {code}
> private void overlay(Properties to, Properties from) {
>  for (Entry<Object, Object> entry: from.entrySet()) {
>  to.put(entry.getKey(), entry.getValue());
>  }
>  }
> {code}
> Properties class is thread-safe but iterator is not. We should manually synchronize on
the returned set of entries which we use for iteration.
> We faced with ResourceManger fails during recovery caused by ConcurrentModificationException:
> {noformat}
> 2018-10-12 08:00:56,968 INFO org.apache.hadoop.service.AbstractService: Service ResourceManager
failed in state STARTED; cause: java.util.ConcurrentModificationException
> java.util.ConcurrentModificationException
>  at java.util.Hashtable$Enumerator.next(Hashtable.java:1383)
>  at org.apache.hadoop.conf.Configuration.overlay(Configuration.java:2801)
>  at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2696)
>  at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2632)
>  at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2528)
>  at org.apache.hadoop.conf.Configuration.get(Configuration.java:1062)
>  at org.apache.hadoop.conf.Configuration.getStringCollection(Configuration.java:1914)
>  at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:53)
>  at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2043)
>  at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2023)
>  at org.apache.hadoop.yarn.webapp.util.WebAppUtils.getPassword(WebAppUtils.java:452)
>  at org.apache.hadoop.yarn.webapp.util.WebAppUtils.loadSslConfiguration(WebAppUtils.java:428)
>  at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:293)
>  at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:1017)
>  at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1117)
>  at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
>  at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1251)
> 2018-10-12 08:00:56,968 INFO org.apache.hadoop.yarn.server.resourcemanager.security.RMDelegationTokenSecretManager:
removing RMDelegation token with sequence number: 3489914
> 2018-10-12 08:00:56,968 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore:
Removing RMDelegationToken and SequenceNumber
> 2018-10-12 08:00:56,968 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore:
Removing RMDelegationToken_3489914
> 2018-10-12 08:00:56,969 INFO org.apache.hadoop.ipc.Server: Stopping server on 8032
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message