ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Speidel" <jspei...@hortonworks.com>
Subject Re: Review Request 35370: JPA persistence failure when adding hosts to an existing config group during blueprint processing
Date Thu, 11 Jun 2015 21:38:00 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35370/
-----------------------------------------------------------

(Updated June 11, 2015, 9:38 p.m.)


Review request for Ambari, Robert Nettleton and Tom Beerbower.


Bugs: AMBARI-11867
    https://issues.apache.org/jira/browse/AMBARI-11867


Repository: ambari


Description (updated)
-------

Adding a host to an existing config group results in a JPA exception. 
This breaks blueprint provisioning/scaling where a host with host group configuration is added
to a cluster.
The BP processor is adding a host to a ConfigGroup instance and persisting it:
group.addHost(host)
group.persist();

This fix is really a band-aid at best.  The JPA code is such a mess that I took the approach
of changing as little as possible to restore functionality.  Even after this fix, there are
still significant issues with the ConfigGroupImpl JPA transaction demarcation and locking
among other things.  I didn't make an attempt to fix these issues here, they will need to
be addressed as part of a larger effor to fix all of the persistence code.  

The "fix" involves calling persistHostMapping() directly instead of calling persist().

I also fixed an issue where it was possible that we could try to add a host to a config group
where it was already registered.


Below is the exception that occurs prior to the fix:

javax.persistence.PersistenceException: java.util.ConcurrentModificationException
at org.eclipse.persistence.internal.jpa.QueryImpl.executeUpdate(QueryImpl.java:308)
at org.apache.ambari.server.orm.dao.DaoUtils.executeUpdate(DaoUtils.java:88)
at org.apache.ambari.server.orm.dao.ConfigGroupConfigMappingDAO.removeAllByGroup(ConfigGroupConfigMappingDAO.java:101)
at org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:60)
at org.apache.ambari.server.state.configgroup.ConfigGroupImpl.persistConfigMapping(ConfigGroupImpl.java:403)
at org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:60)
at org.apache.ambari.server.state.configgroup.ConfigGroupImpl.saveIfPersisted(ConfigGroupImpl.java:460)
at org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:68)
at org.apache.ambari.server.state.configgroup.ConfigGroupImpl.persist(ConfigGroupImpl.java:330)
at org.apache.ambari.server.topology.AmbariContext.addHostToExistingConfigGroups(AmbariContext.java:409)
at org.apache.ambari.server.topology.AmbariContext.registerHostWithConfigGroup(AmbariContext.java:285)
at org.apache.ambari.server.topology.HostRequest$RegisterWithConfigGroupTask.run(HostRequest.java:450)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.ConcurrentModificationException
at java.util.IdentityHashMap$IdentityHashMapIterator.nextIndex(IdentityHashMap.java:734)
at java.util.IdentityHashMap$KeyIterator.next(IdentityHashMap.java:825)
at org.eclipse.persistence.internal.sessions.CollectionChangeRecord.mergeRecord(CollectionChangeRecord.java:260)
at org.eclipse.persistence.internal.sessions.ObjectChangeSet.mergeObjectChanges(ObjectChangeSet.java:740)
at org.eclipse.persistence.internal.sessions.UnitOfWorkChangeSet.mergeObjectChanges(UnitOfWorkChangeSet.java:492)
at org.eclipse.persistence.internal.sessions.UnitOfWorkChangeSet.mergeUnitOfWorkChangeSet(UnitOfWorkChangeSet.java:509)
at org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork.writeChanges(RepeatableWriteUnitOfWork.java:466)
at org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:863)
at org.eclipse.persistence.internal.jpa.QueryImpl.performPreQueryFlush(QueryImpl.java:963)
at org.eclipse.persistence.internal.jpa.QueryImpl.executeUpdate(QueryImpl.java:296)
... 14 more
Exception in thread "pool-3-thread-1" java.lang.RuntimeException: Unable to register config
group for host: c6404.ambari.apache.org
at org.apache.ambari.server.topology.AmbariContext.registerHostWithConfigGroup(AmbariContext.java:290)
at org.apache.ambari.server.topology.HostRequest$RegisterWithConfigGroupTask.run(HostRequest.java:450)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)


Diffs
-----

  ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroup.java
4c806e5 
  ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupImpl.java
d322735 
  ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java 091018a

  ambari-server/src/test/java/org/apache/ambari/server/topology/AmbariContextTest.java 1a234da


Diff: https://reviews.apache.org/r/35370/diff/


Testing
-------

Functional:
Provisioned clusters via a blueprint which previously resulted in the above failure.

Unit:
Updated existing thest and added a new test.
In the progress of running the entire unit test suite and will update with results prior to
merging.


Thanks,

John Speidel


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message