hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oleksandr Shevchenko (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-15865) ConcurrentModificationException in Configuration.overlay() method
Date Fri, 19 Oct 2018 07:51:03 GMT
Oleksandr Shevchenko created HADOOP-15865:
---------------------------------------------

             Summary: ConcurrentModificationException in Configuration.overlay() method
                 Key: HADOOP-15865
                 URL: https://issues.apache.org/jira/browse/HADOOP-15865
             Project: Hadoop Common
          Issue Type: Bug
            Reporter: Oleksandr Shevchenko
            Assignee: Oleksandr Shevchenko


Configuration.overlay() is not thread-safe and can be the cause of ConcurrentModificationException
since we use iteration over Properties object. 
{code}
private void overlay(Properties to, Properties from) {
 for (Entry<Object, Object> entry: from.entrySet()) {
 to.put(entry.getKey(), entry.getValue());
 }
 }
{code}
Properties class is thread-safe but iterator is not. We should manually synchronize on the
returned set of entries which we use for iteration.

We faced with ResourceManger fails during recovery caused by ConcurrentModificationException:
{noformat}
2018-10-12 08:00:56,968 INFO org.apache.hadoop.service.AbstractService: Service ResourceManager
failed in state STARTED; cause: java.util.ConcurrentModificationException
java.util.ConcurrentModificationException
 at java.util.Hashtable$Enumerator.next(Hashtable.java:1383)
 at org.apache.hadoop.conf.Configuration.overlay(Configuration.java:2801)
 at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2696)
 at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2632)
 at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2528)
 at org.apache.hadoop.conf.Configuration.get(Configuration.java:1062)
 at org.apache.hadoop.conf.Configuration.getStringCollection(Configuration.java:1914)
 at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:53)
 at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2043)
 at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2023)
 at org.apache.hadoop.yarn.webapp.util.WebAppUtils.getPassword(WebAppUtils.java:452)
 at org.apache.hadoop.yarn.webapp.util.WebAppUtils.loadSslConfiguration(WebAppUtils.java:428)
 at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:293)
 at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:1017)
 at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1117)
 at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
 at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1251)
2018-10-12 08:00:56,968 INFO org.apache.hadoop.yarn.server.resourcemanager.security.RMDelegationTokenSecretManager:
removing RMDelegation token with sequence number: 3489914
2018-10-12 08:00:56,968 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore:
Removing RMDelegationToken and SequenceNumber
2018-10-12 08:00:56,968 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore:
Removing RMDelegationToken_3489914
2018-10-12 08:00:56,969 INFO org.apache.hadoop.ipc.Server: Stopping server on 8032
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message