accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Calvin Feder <calvin.fe...@argyledata.com>
Subject RE: Issue launching accumulo proxy
Date Thu, 07 May 2015 16:58:59 GMT
Thank you Sean for your feedback.

Do we need to have a gateway role defined?  What does this gateway role do?  Should it be
defined on the cloudera manager server, or on the server which is running the proxy?  I didn’t
have a gateway role defined before, but I have now defined a gateway role on the cloudera
manager server node (which is also an accumulo node, as this is only a 3 node cluster).  The
error still appears.  Do I need to restart accumulo for that change to take effect?

We have tried with and without defining the ACCUMULO_HOME, and we have tried with the full
path to the accumulo command and using the one in the path (which is actually just a sim link
into the cloudera parcels).  The same error appears regardless.

For the zookeepers, we do not have a fully qualified domain name in this cluster.  There is
actually no DNS server, and these names resolve because they are in the hosts file on each
machine.  I could try using IP addresses instead, but I don’t think that is related to my
problem.

Thanks for the info about cloudera being able to remove and re-write that directory.  We actually
are no longer using that jar and I simply forgot it was there.

Thanks,
Calvin

From: Sean Busbey [mailto:busbey@cloudera.com]
Sent: Wednesday, May 06, 2015 2:59 PM
To: Accumulo User List
Subject: Re: Issue launching accumulo proxy

A few questions / statements

1) Does this node have a Gateway role defined?

2) Definitely don't define ACCUMULO_HOME. all the environment variables get set by CM for
the "accumulo" command that is placed in your PATH for a gateway node.

3) When I use the accumulo command from the path instead of looking in the parcel, the proxy
works (after I update the connection for my cluster)

I suspect that you are using "accumulo classpath" for the classpath check, which is the CM
deployed gateway command, and then using $ACCUMULO_HOME/bin/accumulo for the proxy, which
is not the same thing.

As a side note, I noticed this bit:

> zookeepers=argyle1:2181,argyle2:2181,argyl3:2181

You should use fully qualified domain names.

>  Level 4: Accumulo Dynamic Classloader (loads everything defined by general.dynamic.classpaths)
VFS classpaths items are:
        jar:file:///opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/ext/fraudProcessor-1.0.0-20141028.170419-25.jar!/

The parcel directory should be treated as read-only. CM may remove and rewrite this directory,
so it's not safe to keep your jars there for a deployment.


On Wed, May 6, 2015 at 1:58 PM, Calvin Feder <calvin.feder@argyledata.com<mailto:calvin.feder@argyledata.com>>
wrote:

Yes it does include that class.



I will try raising the issue with Cloudera.



Thanks,

Calvin

________________________________
From: Christopher <ctubbsii@apache.org<mailto:ctubbsii@apache.org>>
Sent: Wednesday, May 06, 2015 1:54 PM
To: Accumulo User List
Subject: Re: Issue launching accumulo proxy

Does the output of:
jar tf /opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/accumulo-core.jar
include the following file?
org/apache/accumulo/core/client/security/tokens/PasswordToken.class


--
Christopher L Tubbs II
http://gravatar.com/ctubbsii

On Wed, May 6, 2015 at 4:12 PM, Calvin Feder <calvin.feder@argyledata.com<mailto:calvin.feder@argyledata.com>>
wrote:
We are trying to launch the accumulo thrift proxy (accumulo 1.6) in a cluster deployed through
Cloudera Manager version 5.1.  We seem to be hitting a classpath issue, yet the classpath
appears to be correct.

This is the error we see:

/opt/cloudera/parcels/ACCUMULO/bin/accumulo proxy -p /opt/argyledata/argyledb/proxy.properties
Thread "proxy" died null
java.lang.reflect.InvocationTargetException
       at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
       at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
       at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
       at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
       at org.apache.accumulo.proxy.Proxy.createProxyServer(Proxy.java:128)
       at org.apache.accumulo.proxy.Proxy.main(Proxy.java:119)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:606)
       at org.apache.accumulo.start.Main$1.run(Main.java:141)
       at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.accumulo.core.client.security.tokens.PasswordToken
       at org.apache.accumulo.proxy.ProxyServer.<init>(ProxyServer.java:185)
       ... 12 more
Caused by: java.lang.ClassNotFoundException: org.apache.accumulo.core.client.security.tokens.PasswordToken
       at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
       at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
       at java.security.AccessController.doPrivileged(Native Method)
       at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
       at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
       at org.apache.accumulo.start.classloader.AccumuloClassLoader$2.loadClass(AccumuloClassLoader.java:275)
       at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
       at java.lang.Class.forName0(Native Method)
       at java.lang.Class.forName(Class.java:190)
       at org.apache.accumulo.proxy.ProxyServer.<init>(ProxyServer.java:183)
       ... 12 more

This is the content of proxy.properties:

protocolFactory=org.apache.thrift.protocol.TBinaryProtocol$Factory
tokenClass=org.apache.accumulo.core.client.security.tokens.PasswordToken
port=42425
instance=accumulo
zookeepers=argyle1:2181,argyle2:2181,argyl3:2181


And this is the output of the command “accumulo classpath”:

Level 1: Java System Classloader (loads Java system resources) URL classpath items are:
        file:/usr/java/jdk1.7.0_65/jre/lib/ext/sunjce_provider.jar
        file:/usr/java/jdk1.7.0_65/jre/lib/ext/sunec.jar
        file:/usr/java/jdk1.7.0_65/jre/lib/ext/localedata.jar
        file:/usr/java/jdk1.7.0_65/jre/lib/ext/sunpkcs11.jar
        file:/usr/java/jdk1.7.0_65/jre/lib/ext/dnsns.jar
        file:/usr/java/jdk1.7.0_65/jre/lib/ext/zipfs.jar

Level 2: Java Classloader (loads everything defined by java classpath) URL classpath items
are:
        file:/etc/accumulo/conf.cloudera.accumulo16/
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/accumulo-start.jar
        file:/opt/cloudera/parcels/CDH-5.1.0-1.cdh5.1.0.p0.53/lib/hadoop/lib/log4j-1.2.17.jar

Level 3: Accumulo Classloader (loads everything defined by general.classpaths) URL classpath
items are:
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/accumulo-examples-simple.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/commons-vfs2.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/guava.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/jetty-util.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/accumulo-server-base.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/jetty.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/commons-math.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/jcommander.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/accumulo-tserver.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/accumulo-master.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/accumulo-tracer.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/accumulo-monitor.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/accumulo-gc.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/libthrift.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/accumulo-start.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/accumulo-core.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/accumulo-proxy.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/jline.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/accumulo-test.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/gson.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/servlet-api.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/accumulo-minicluster.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/accumulo-fate.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/accumulo-trace.jar
        file:/etc/hadoop/conf/
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/commons-lang-2.6.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/commons-collections-3.2.1.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/commons-cli-1.2.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/commons-httpclient-3.1.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/commons-compress-1.4.1.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/jsr305-1.3.9.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/commons-logging-1.1.3.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/protobuf-java-2.5.0.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/paranamer-2.3.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/commons-math3-3.1.1.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/snappy-java-1.0.4.1.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/jackson-core-asl-1.8.8.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/commons-digester-1.8.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/commons-configuration-1.6.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/jersey-core-1.9.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/activation-1.1.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/xz-1.0.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/jetty-util-6.1.26.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/httpclient-4.2.5.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/commons-beanutils-1.7.0.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/httpcore-4.2.5.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/commons-codec-1.4.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/jaxb-api-2.2.2.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/servlet-api-2.5.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/xmlenc-0.52.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/commons-beanutils-core-1.8.0.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/jackson-mapper-asl-1.8.8.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/slf4j-api-1.7.5.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/commons-io-2.4.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/log4j-1.2.17.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/stax-api-1.0-2.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/commons-net-3.1.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/slf4j-log4j12.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/avro.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/hadoop-yarn-client-2.3.0-cdh5.1.0.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/hadoop-yarn-common-2.3.0-cdh5.1.0.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/hadoop-mapreduce-client-core-2.3.0-cdh5.1.0.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/hadoop-hdfs-2.3.0-cdh5.1.0.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/hadoop-mapreduce-client-jobclient-2.3.0-cdh5.1.0.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/hadoop-mapreduce-client-shuffle-2.3.0-cdh5.1.0.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/hadoop-mapreduce-client-common-2.3.0-cdh5.1.0.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/hadoop-mapreduce-client-app-2.3.0-cdh5.1.0.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/hadoop-common-2.3.0-cdh5.1.0.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/hadoop-annotations-2.3.0-cdh5.1.0.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/hadoop-yarn-server-common-2.3.0-cdh5.1.0.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/hadoop-yarn-api-2.3.0-cdh5.1.0.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/hadoop/client/hadoop-auth-2.3.0-cdh5.1.0.jar
        file:/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/bin/../lib/../../CDH/lib/zookeeper/zookeeper-3.4.5-cdh5.1.0.jar

Level 4: Accumulo Dynamic Classloader (loads everything defined by general.dynamic.classpaths)
VFS classpaths items are:
        jar:file:///opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo/lib/ext/fraudProcessor-1.0.0-20141028.170419-25.jar!/


We have even tried setting accumulo home in the context of the proxy start command:

ACCUMULO_HOME=/opt/cloudera/parcels/ACCUMULO-1.6.0-1.cdh5.1.0.p0.51/lib/accumulo



We greatly appreciate any assistance.

___________________________________________
Calvin Feder | QA Engineer | Argyle Data, Inc.
[cid:A79ABAC4-829A-4B95-AFE2-C09282CD7AE7]




________________________________

This message is for the designated recipient only and may contain privileged, proprietary,
or otherwise confidential information. If you have received it in error, please notify the
sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Thank you in advance for your cooperation.

________________________________



________________________________

This message is for the designated recipient only and may contain privileged, proprietary,
or otherwise confidential information. If you have received it in error, please notify the
sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Thank you in advance for your cooperation.

________________________________


________________________________

This message is for the designated recipient only and may contain privileged, proprietary,
or otherwise confidential information. If you have received it in error, please notify the
sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Thank you in advance for your cooperation.

________________________________



--
Sean


________________________________

This message is for the designated recipient only and may contain privileged, proprietary,
or otherwise confidential information. If you have received it in error, please notify the
sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Thank you in advance for your cooperation.

________________________________



________________________________

This message is for the designated recipient only and may contain privileged, proprietary,
or otherwise confidential information. If you have received it in error, please notify the
sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Thank you in advance for your cooperation.

________________________________
Mime
View raw message