atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pierre Padovani <pie...@padovani.org>
Subject Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas
Date Mon, 26 Mar 2018 19:37:18 GMT


> On March 26, 2018, 1:32 p.m., David Radley wrote:
> > pom.xml
> > Line 673 (original), 673 (patched)
> > <https://reviews.apache.org/r/65885/diff/4/?file=1982751#file1982751line673>
> >
> >     I did a build with  :"mvn clean  package -Pdist" without your patch and it runs
through cleanly.
> >     
> >     I apply your patch and build with "mvn clean  package -Pdist,embedded-cassandra-solr"
and get a build failure - details below. During the integration tests, it tries to spin up
an Atlas and an embedded solr. This works slightly differently depending on the build options.
I suspect that there is something in the integration tests for the embedded cassandra build
that is not set up correctly so it is not fidning the embdedd solr.     
> >     
> >      ------------------------------------------------------------------------
> >     [INFO] BUILD FAILURE
> >     [INFO] ------------------------------------------------------------------------
> >     [INFO] Total time: 04:31 min
> >     [INFO] Finished at: 2018-03-26T13:49:19+01:00
> >     [INFO] Final Memory: 146M/1574M
> >     [INFO] ------------------------------------------------------------------------
> >     [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test
(default-test) on project atlas-repository: ExecutionException: java.lang.RuntimeException:
There was an error in the forked process
> >     [ERROR] java.lang.ExceptionInInitializerError
> >     [ERROR] 	at java.lang.Class.forName0(Native Method)
> >     [ERROR] 	at java.lang.Class.forName(Class.java:264)
> >     [ERROR] 	at org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:132)
> >     [ERROR] 	at org.apache.atlas.util.AtlasRepositoryConfiguration.getDeleteHandlerV1Impl(AtlasRepositoryConfiguration.java:98)
> >     [ERROR] 	at org.apache.atlas.TestModules$TestOnlyModule.bindDeleteHandler(TestModules.java:160)
> >     [ERROR] 	at org.apache.atlas.TestModules$TestOnlyModule.configure(TestModules.java:115)
> >     [ERROR] 	at com.google.inject.AbstractModule.configure(AbstractModule.java:62)
> >     [ERROR] 	at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
> >     [ERROR] 	at com.google.inject.spi.Elements.getElements(Elements.java:110)
> >     [ERROR] 	at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
> >     [ERROR] 	at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
> >     [ERROR] 	at com.google.inject.internal.InjectorImpl.createChildInjector(InjectorImpl.java:232)
> >     [ERROR] 	at org.testng.TestRunner.getInjector(TestRunner.java:1626)
> >     [ERROR] 	at org.testng.internal.ClassImpl.getInstanceFromGuice(ClassImpl.java:132)
> >     [ERROR] 	at org.testng.internal.ClassImpl.getDefaultInstance(ClassImpl.java:112)
> >     [ERROR] 	at org.testng.internal.ClassImpl.getInstances(ClassImpl.java:183)
> >     [ERROR] 	at org.testng.internal.TestNGClassFinder.<init>(TestNGClassFinder.java:128)
> >     [ERROR] 	at org.testng.TestRunner.initMethods(TestRunner.java:413)
> >     [ERROR] 	at org.testng.TestRunner.init(TestRunner.java:239)
> >     [ERROR] 	at org.testng.TestRunner.init(TestRunner.java:209)
> >     [ERROR] 	at org.testng.TestRunner.<init>(TestRunner.java:157)
> >     [ERROR] 	at org.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:546)
> >     [ERROR] 	at org.testng.SuiteRunner.init(SuiteRunner.java:159)
> >     [ERROR] 	at org.testng.SuiteRunner.<init>(SuiteRunner.java:113)
> >     [ERROR] 	at org.testng.TestNG.createSuiteRunner(TestNG.java:1274)
> >     [ERROR] 	at org.testng.TestNG.createSuiteRunners(TestNG.java:1261)
> >     [ERROR] 	at org.testng.TestNG.runSuitesLocally(TestNG.java:1115)
> >     [ERROR] 	at org.testng.TestNG.run(TestNG.java:1032)
> >     [ERROR] 	at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:115)
> >     [ERROR] 	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:129)
> >     [ERROR] 	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:113)
> >     [ERROR] 	at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:111)
> >     [ERROR] 	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
> >     [ERROR] 	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
> >     [ERROR] 	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
> >     [ERROR] Caused by: java.lang.RuntimeException: Failed to start embedded solr
cloud server. Aborting!
> >     [ERROR] 	at org.apache.atlas.repository.graphdb.janus.AtlasJanusGraphDatabase.startLocalSolr(AtlasJanusGraphDatabase.java:199)
> >     [ERROR] 	at org.apache.atlas.repository.graphdb.janus.AtlasJanusGraphDatabase.getConfiguration(AtlasJanusGraphDatabase.java:74)
> >     [ERROR] 	at org.apache.atlas.repository.graphdb.janus.AtlasJanusGraphDatabase.getGraphInstance(AtlasJanusGraphDatabase.java:105)
> >     [ERROR] 	at org.apache.atlas.repository.graphdb.janus.AtlasJanusGraphDatabase.getGraph(AtlasJanusGraphDatabase.java:186)
> >     [ERROR] 	at org.apache.atlas.repository.graph.AtlasGraphProvider.getGraphInstance(AtlasGraphProvider.java:40)
> >     [ERROR] 	at org.apache.atlas.repository.graph.GraphHelper.getInstance(GraphHelper.java:122)
> >     [ERROR] 	at org.apache.atlas.repository.store.graph.v1.DeleteHandlerV1.<clinit>(DeleteHandlerV1.java:72)
> >     [ERROR] 	... 35 more
> >     [ERROR] Caused by: java.lang.Exception: Error starting up MiniSolrCloudCluster
> >     [ERROR] 	at org.apache.solr.cloud.MiniSolrCloudCluster.checkForExceptions(MiniSolrCloudCluster.java:507)
> >     [ERROR] 	at org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:251)
> >     [ERROR] 	at org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:195)
> >     [ERROR] 	at org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:179)
> >     [ERROR] 	at org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:160)
> >     [ERROR] 	at org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:140)
> >     [ERROR] 	at org.apache.atlas.runner.LocalSolrRunner.start(LocalSolrRunner.java:63)
> >     [ERROR] 	at org.apache.atlas.repository.graphdb.janus.AtlasJanusGraphDatabase.startLocalSolr(AtlasJanusGraphDatabase.java:193)
> >     [ERROR] 	... 41 more
> >     [ERROR] 	Suppressed: java.lang.NoClassDefFoundError: com/codahale/metrics/MetricRegistry$MetricSupplier
> >     [ERROR] 		at org.apache.solr.metrics.SolrMetricManager.<init>(SolrMetricManager.java:119)
> >     [ERROR] 		at org.apache.solr.core.CoreContainer.load(CoreContainer.java:492)
> >     [ERROR] 		at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:261)
> >     [ERROR] 		at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:181)
> >     [ERROR] 		at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
> >     [ERROR] 		at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
> >     [ERROR] 		at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
> >     [ERROR] 		at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
> >     [ERROR] 		at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
> >     [ERROR] 		at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
> >     [ERROR] 		at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
> >     [ERROR] 		at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
> >     [ERROR] 		at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
> >     [ERROR] 		at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
> >     [ERROR] 		at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
> >     [ERROR] 		at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
> >     [ERROR] 		at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:384)
> >     [ERROR] 		at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$4(MiniSolrCloudCluster.java:247)
> >     [ERROR] 		at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >     [ERROR] 		at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$128(ExecutorUtil.java:188)
> >     [ERROR] 		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >     [ERROR] 		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >     [ERROR] 		at java.lang.Thread.run(Thread.java:748)
> >     [ERROR] 	Caused by: java.lang.ClassNotFoundException: com.codahale.metrics.MetricRegistry$MetricSupplier
> >     [ERROR] 		at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> >     [ERROR] 		at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> >     [ERROR] 		at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
> >     [ERROR] 		at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> >     [ERROR] 		... 23 more
> >     [ERROR] -> [Help 1]
> >     [ERROR] 
> >     [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
> >     [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> >     [ERROR] 
> >     [ERROR] For more information about the errors and possible solutions, please
read the following articles:
> >     [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> >     [ERROR] 
> >     [ERROR] After correcting the problems, you can resume the build with the command
> >     [ERROR]   mvn <goals> -rf :atlas-repository
> 
> Pierre Padovani wrote:
>     I just attempted a build on master using 'mvn clean package -Pdist' and got the same
failure. I do not think this failure is due to my patch. I'm running on OS X, what are you
using?

Ok... got this one figured out. Looks like the storm-bridge/shim is pulling in an old version
of the 'io.dropwizard.metrics:metrics-core'. The order of classpath loading may differ on
a system based on CPU speed etc. So I'm guessing that this is why you didn't see it and I
did. This same dependency is introduced with the janus-cassandra inclusion, so I'm going to
fix this across the board to make sure we are only using/including one version of these metric
jars.


- Pierre


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review199964
-----------------------------------------------------------


On March 20, 2018, 12:04 a.m., Pierre Padovani wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65885/
> -----------------------------------------------------------
> 
> (Updated March 20, 2018, 12:04 a.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Bugs: ATLAS-2470
>     https://issues.apache.org/jira/browse/ATLAS-2470
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> This updates pom's to add the needed cassandra jars, and adds a dist profile embedded-cassandra-solr.
> 
> 
> Diffs
> -----
> 
>   distro/pom.xml 0103bef6 
>   distro/src/bin/atlas_config.py e6415cf4 
>   distro/src/bin/atlas_start.py 39be6b7c 
>   distro/src/bin/atlas_stop.py 66edd904 
>   distro/src/conf/atlas-application.properties 10fb7b86 
>   distro/src/conf/atlas-env.sh 68b24e93 
>   distro/src/conf/cassandra.yml.template PRE-CREATION 
>   distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
>   distro/src/conf/zookeeper/zoo.cfg.template PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 1881082e 
>   docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
>   graphdb/janus/pom.xml 143b775f 
>   pom.xml 9ddc8e18 
>   repository/pom.xml b1d6b1f9 
>   repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
cee7374e 
>   repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
4d70b7f7 
>   repository/src/test/java/org/apache/atlas/repository/audit/CassandraAuditRepositoryTest.java
PRE-CREATION 
>   repository/src/test/resources/cassandra_test.yml PRE-CREATION 
>   server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java 7d282b30

> 
> 
> Diff: https://reviews.apache.org/r/65885/diff/4/
> 
> 
> Testing
> -------
> 
> Full build with the new embedded-cassandra-solr, and testing to make sure Atlas comes
up and is functional.
> 
> Be aware that we have been running Cassandra backed JanusGraph for months with no issues.
> 
> This patch now includes support for Cassandra as an audit store. I've also refactored
the audit unit tests, and added cassandra specific tests.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message