atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Radley <david_rad...@uk.ibm.com>
Subject Fw: Review Request 63892: [ATLAS-2264]: Update JanusGraph version to 0.2.0
Date Mon, 20 Nov 2017 14:13:54 GMT
Hi Sarath,
Last weekends checkins broke me,  I found a way to set the configuration 
so that titan0 worked at the end of last week. This weekend there were 
more checkins (to bring in Janus v2) and I used the same method to build 
berkeley elastic search on titan0 and am broken again :-(. The log is 
below. 
I tried building with berkeley and Titan0 (without elastic search) - it 
did generate the config with solr, bu it did not include berkeley classes 
in the build - so cannot find them even if you specify 
the right configuration. 

Note that in all these failing cases it looks like Atlas has started - but 
Atlas will not accept any Rest calls. 



13:54 $ ls
application.log                 atlas.20171120-135419.err 
atlas.20171120-135419.out       atlas.pid                       audit.log  
        metric.log
✔ 
~/atlas/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/logs 
[master|●108✚ 3] 
13:54 $ vi atlas.20171120-135419.out

2017-11-20 13:54:20,204 WARN  - [main:] ~ Using @Deprecated Class 
org.springframework.web.util.Log4jConfigListener (DeprecationWarning:43)
2017-11-20 13:54:23,161 WARN  - [main:] ~ Exception encountered during 
context initialization - cancelling refresh attempt: 
org.springframework.beans.factory.BeanCreationException: Error creating 
bean with name 
'org.springframework.context.event.internalEventListenerProcessor': 
BeanPostProcessor before instantiation of bean failed; nested exception is 
org.springframework.beans.factory.UnsatisfiedDependencyException: Error 
creating bean with name 'graphTransactionAdvisor' defined in URL 
[jar:file:/Users/david/atlas/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0-SNAPSHOT.jar!/org/apache/atlas/GraphTransactionAdvisor.class]:

Unsatisfied dependency expressed through constructor parameter 0; nested 
exception is 
org.springframework.beans.factory.UnsatisfiedDependencyException: Error 
creating bean with name 'graphTransactionInterceptor' defined in URL 
[jar:file:/Users/david/atlas/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0-SNAPSHOT.jar!/org/apache/atlas/GraphTransactionInterceptor.class]:

Unsatisfied dependency expressed through constructor parameter 0; nested 
exception is org.springframework.beans.factory.BeanCreationException: 
Error creating bean with name 'get' defined in 
org.apache.atlas.repository.graph.AtlasGraphProvider: Bean instantiation 
via factory method failed; nested exception is 
org.springframework.beans.BeanInstantiationException: Failed to 
instantiate [org.apache.atlas.repository.graphdb.AtlasGraph]: Factory 
method 'get' threw exception; nested exception is 
java.lang.IllegalArgumentException: Could not instantiate implementation: 
com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex 
(AbstractApplicationContext:550)
2017-11-20 13:54:23,163 ERROR - [main:] ~ Context initialization failed 
(ContextLoader:350)
org.springframework.beans.factory.BeanCreationException: Error creating 
bean with name 
'org.springframework.context.event.internalEventListenerProcessor': 
BeanPostProcessor before instantiation of bean failed; nested exception is 
org.springframework.beans.factory.UnsatisfiedDependencyException: Error 
creating bean with name 'graphTransactionAdvisor' defined in URL 
[jar:file:/Users/david/atlas/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0-SNAPSHOT.jar!/org/apache/atlas/GraphTransactionAdvisor.class]:

Unsatisfied dependency expressed through constructor parameter 0; nested 
exception is 
org.springframework.beans.factory.UnsatisfiedDependencyException: Error 
creating bean with name 'graphTransactionInterceptor' defined in URL 
[jar:file:/Users/david/atlas/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0-SNAPSHOT.jar!/org/apache/atlas/GraphTransactionInterceptor.class]:

Unsatisfied dependency expressed through constructor parameter 0; nested 
exception is org.springframework.beans.factory.BeanCreationException: 
Error creating bean with name 'get' defined in 
org.apache.atlas.repository.graph.AtlasGraphProvider: Bean instantiation 
via factory method failed; nested exception is 
org.springframework.beans.BeanInstantiationException: Failed to 
instantiate [org.apache.atlas.repository.graphdb.AtlasGraph]: Factory 
method 'get' threw exception; nested exception is 
java.lang.IllegalArgumentException: Could not instantiate implementation: 
com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex
        at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:479)
        at 
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
        at 
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
        at 
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
        at 
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
        at 
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
        at 
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:866)
        at 
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:542)
        at 
org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)
        at 
org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325)
        at 
org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
        at 
org.apache.atlas.web.setup.KerberosAwareListener.contextInitialized(KerberosAwareListener.java:31)
        at 
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:843)
        at 
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:533)
        at 
org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:816)
        at 
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:345)
        at 
org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1404)
        at 
org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1366)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:778)
        at 
org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:262)
        at 
org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:520)
        at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
        at org.eclipse.jetty.server.Server.start(Server.java:422)
        at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
        at 
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
        at org.eclipse.jetty.server.Server.doStart(Server.java:389)
        at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at 
org.apache.atlas.web.service.EmbeddedServer.start(EmbeddedServer.java:98)
        at org.apache.atlas.Atlas.main(Atlas.java:133)
Caused by: 
org.springframework.beans.factory.UnsatisfiedDependencyException: Error 
creating bean with name 'graphTransactionAdvisor' defined in URL 
[jar:file:/Users/david/atlas/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0-SNAPSHOT.jar!/org/apache/atlas/GraphTransactionAdvisor.class]:

Unsatisfied dependency expressed through constructor parameter 0; nested 
exception is 
org.springframework.beans.factory.UnsatisfiedDependencyException: Error 
creating bean with name 'graphTransactionInterceptor' defined in URL 
[jar:file:/Users/david/atlas/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/server/webapp/atlas/WEB-INF/lib/atlas-repository-1.0.0-SNAPSHOT.jar!/org/apache/atlas/GraphTransactionInterceptor.class]:

Unsatisfied dependency expressed through constructor parameter 0; nested 
exception is org.springframework.beans.factory.BeanCreationException: 
Error creating bean with name 'get' defined in 
org.apache.atlas.repository.graph.AtlasGraphProvider: Bean instantiation 
via factory method failed; nested exception is 
org.springframework.beans.BeanInstantiationException: Failed to 
instantiate [org.apache.atlas.repository.graphdb.AtlasGraph]: Factory 
method 'get' threw exception; nested exception is 
java.lang.IllegalArgumentException: Could not instantiate implementation: 
com.thinkaurelius.titan.diskstorage.es.ElasticSearchIndex
        at 
org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:749)
        at 
org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:189)
        at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1193)
        at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1095)
        at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:513)
        at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
        at 
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
        at 
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
        at 
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
        at 
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
        at 
org.springframework.aop.framework.autoproxy.BeanFactoryAdvisorRetrievalHelper.findAdvisorBeans(BeanFactoryAdvisorRetrievalHelper.java:92)
        at 
org.springframework.aop.framework.autoproxy.AbstractAdvisorAutoProxyCreator.findCandidateAdvisors(AbstractAdvisorAutoProxyCreator.java:102)
        at 
org.springframework.aop.aspectj.autoproxy.AspectJAwareAdvisorAutoProxyCreator.shouldSkip(AspectJAwareAdvisorAutoProxyCreator.java:103)
        at 
org.springframework.aop.framework.autoproxy.AbstractAutoProxyCreator.postProcessBeforeInstantiation(AbstractAutoProxyCreator.java:248)
        at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInstantiation(AbstractAutowireCapableBeanFactory.java:1037)
        at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.resolveBeforeInstantiation(AbstractAutowireCapableBeanFactory.java:1011)
        at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:473)
        ... 29 more
@                                                     @                    
                                                           @               
  
 
all the best , David. 


----- Forwarded by David Radley/UK/IBM on 20/11/2017 13:55 -----

From:   David Radley/UK/IBM
To:     "Sarath Subramanian" <sarath@apache.org>
Cc:     dev@atlas.apache.org
Date:   20/11/2017 12:25
Subject:        Re: Review Request 63892: [ATLAS-2264]: Update JanusGraph 
version to 0.2.0


Hi Sarath,
I have done a git pull.
1) I ran mvn clean,package which compiles.
2) I start atlas it seems to start - but it will not accept any rest 
calls. the log shows it cannot find the janus class. 
3) I configuration file that was generated had berkleyje as the backend. I 
changed this to hbase and then get
 Exception: ('Could not find hbase-site.xml in %s. Please set env var 
HBASE_CONF_DIR to the hbase client conf dir', 
'/Users/david/atlas/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/hbase/conf')


Traceback (most recent call last):
  File 
"/Users/david/atlas/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/bin/atlas_start.py",

line 149, in <module>
    returncode = main()
  File 
"/Users/david/atlas/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/bin/atlas_start.py",

line 92, in main
    raise Exception("Could not find hbase-site.xml in %s. Please set env 
var HBASE_CONF_DIR to the hbase client conf dir", hbase_conf_dir)
Exception: ('Could not find hbase-site.xml in %s. Please set env var 
HBASE_CONF_DIR to the hbase client conf dir', 
'/Users/david/atlas/distro/target/apache-atlas-1.0.0-SNAPSHOT-bin/apache-atlas-1.0.0-SNAPSHOT/hbase/conf')
 
I was expecting by default that the config file would be janus backed by 
hbase and that it would use the hbase that was under Janus - so I did not 
need to configure anything extra. 

Any suggestions on how I can get Atlas working with Janus?   all the best, 
David. 




From:   Sarath Subramanian <sarath@apache.org>
To:     Apoorv Naik <naik.apoorv@gmail.com>, Madhan Neethiraj 
<madhan@apache.org>, Ashutosh Mestry <amestry@hortonworks.com>
Cc:     atlas <dev@atlas.incubator.apache.org>, Sarath Subramanian 
<sarath@apache.org>
Date:   16/11/2017 23:12
Subject:        Review Request 63892: [ATLAS-2264]: Update JanusGraph 
version to 0.2.0
Sent by:        Sarath Subramanian <noreply@reviews.apache.org>




-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://urldefense.proofpoint.com/v2/url?u=https-3A__reviews.apache.org_r_63892_&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=QhpUQPr5YlG95aAgCvZGStEXHg4hBbSYQ9JkRqR_svY&m=c6zcTJP3lgR7Ao9dXHdZjpTUwWwdTfIOVcAj8JFuSqU&s=7H6x2smMBVHJoW1E4_c-iIsQssgc3wHMsAaeS9SU-F0&e=

-----------------------------------------------------------

Review request for atlas, Apoorv Naik, Ashutosh Mestry, and Madhan 
Neethiraj.


Bugs: ATLAS-2264
    
https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_ATLAS-2D2264&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=QhpUQPr5YlG95aAgCvZGStEXHg4hBbSYQ9JkRqR_svY&m=c6zcTJP3lgR7Ao9dXHdZjpTUwWwdTfIOVcAj8JFuSqU&s=RdYkPd0KSq8QQAEcPuGI2HrG85KrOGUCgYgnW3dNAYM&e=



Repository: atlas


Description
-------

Currently Atlas uses JanusGraph 0.1.1. This doesn't have support for Solr 
6.x and other fixes. Atlas needs to use the latest JanusGraph release 
version.
JanusGraph 0.2.0 doesn't package local elasticsearch or solr and only 
provides mechanism to connect to a running instance
This JIRA addresses startup/shutdown of a local solr cloud instance for 
use in Atlas unit tests


Diffs
-----

  addons/falcon-bridge-shim/pom.xml 6cbaf0ad 
  addons/falcon-bridge/pom.xml 68ddec1a 
  addons/hbase-bridge-shim/pom.xml 6d01a3e6 
  addons/hbase-bridge/pom.xml 3dc4f1b7 
  addons/hive-bridge/pom.xml 42af8bb3 
  addons/sqoop-bridge/pom.xml 9c88a865 
  addons/storm-bridge-shim/pom.xml f20a1abb 
  addons/storm-bridge/pom.xml 4908ccf4 
  common/pom.xml 6d7515b0 
  common/src/test/java/org/apache/atlas/runner/LocalSolrRunner.java 
PRE-CREATION 
  common/src/test/java/org/apache/atlas/runner/LocalSolrTest.java 
PRE-CREATION 
  common/src/test/resources/log4j.properties PRE-CREATION 
  common/src/test/resources/solr/collections.txt PRE-CREATION 
  common/src/test/resources/solr/core-template/currency.xml PRE-CREATION 
  common/src/test/resources/solr/core-template/lang/stopwords_en.txt 
PRE-CREATION 
  common/src/test/resources/solr/core-template/protwords.txt PRE-CREATION 
  common/src/test/resources/solr/core-template/schema.xml PRE-CREATION 
  common/src/test/resources/solr/core-template/solrconfig.xml PRE-CREATION 

  common/src/test/resources/solr/core-template/stopwords.txt PRE-CREATION 
  common/src/test/resources/solr/core-template/synonyms.txt PRE-CREATION 
  common/src/test/resources/solr/solr.xml PRE-CREATION 
 graphdb/common/src/test/java/org/apache/atlas/graph/GraphSandboxUtil.java 
3cccd84e 
  graphdb/janus/pom.xml ad96db3f 
 
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java

7c96806d 
 
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphDatabase.java

1fda689d 
 
graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/AbstractGraphDatabaseTest.java

5574a4ce 
 
graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusDatabaseTest.java

4919de24 
 
graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/JanusGraphProviderTest.java

7c1dda7b 
  graphdb/janus/src/test/resources/atlas-application.properties a66e8651 
  graphdb/titan0/pom.xml 2eec260d 
  graphdb/titan1/pom.xml c729caa6 
  intg/pom.xml 22895df7 
  intg/src/main/resources/atlas-log4j.xml PRE-CREATION 
  intg/src/test/resources/atlas-application.properties b937c33a 
  pom.xml cbf9e2af 
  repository/pom.xml 51ddb763 
  repository/src/test/java/org/apache/atlas/TestModules.java 5c9e60f6 
 
repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java 
47fd9cfc 
 
repository/src/test/java/org/apache/atlas/repository/impexp/ExportServiceTest.java 
2a8bdfae 
 
repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java 
b24774d6 
 
repository/src/test/java/org/apache/atlas/repository/impexp/LocalSolrTest.java 
PRE-CREATION 
 
repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java

9088470e 
 
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityDefStoreV1Test.java

11982830 
 
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java

fd1b6db0 
 
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipDefStoreV1Test.java

59bedd12 
 
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java

d207a695 
 
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java

2c21638f 
 
repository/src/test/java/org/apache/atlas/repository/userprofile/UserProfileServiceTest.java

0532f161 
 
repository/src/test/java/org/apache/atlas/services/MetricsServiceTest.java 
5165bcb9 
  repository/src/test/resources/solr/collections.txt PRE-CREATION 
  repository/src/test/resources/solr/core-template/currency.xml 
PRE-CREATION 
  repository/src/test/resources/solr/core-template/lang/stopwords_en.txt 
PRE-CREATION 
  repository/src/test/resources/solr/core-template/protwords.txt 
PRE-CREATION 
  repository/src/test/resources/solr/core-template/schema.xml PRE-CREATION 

  repository/src/test/resources/solr/core-template/solrconfig.xml 
PRE-CREATION 
  repository/src/test/resources/solr/core-template/stopwords.txt 
PRE-CREATION 
  repository/src/test/resources/solr/core-template/synonyms.txt 
PRE-CREATION 
  repository/src/test/resources/solr/solr.xml PRE-CREATION 
  server-api/pom.xml 6919dcdd 
  shaded/hbase-server-shaded/pom.xml 3cf500a8 
  webapp/pom.xml 569e06f8 


Diff: 
https://urldefense.proofpoint.com/v2/url?u=https-3A__reviews.apache.org_r_63892_diff_1_&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=QhpUQPr5YlG95aAgCvZGStEXHg4hBbSYQ9JkRqR_svY&m=c6zcTJP3lgR7Ao9dXHdZjpTUwWwdTfIOVcAj8JFuSqU&s=Ihplz6iFEDHUdFvkduwI9KopSGYC8cmbCfxm5d8sX1g&e=



Testing
-------

mvn clean package => builds janus profile and start/shutdown of local solr 
cluster for unit tests

mvn clean package -DGRAPH-PROVIDER=titan0  => uses titan0 profile and use 
elasticsearch packaged with titan jar


Thanks,

Sarath Subramanian




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message