Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E98B5180DE for ; Thu, 16 Jul 2015 14:27:00 +0000 (UTC) Received: (qmail 31417 invoked by uid 500); 16 Jul 2015 14:26:55 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 31366 invoked by uid 500); 16 Jul 2015 14:26:55 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 31355 invoked by uid 99); 16 Jul 2015 14:26:55 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 16 Jul 2015 14:26:55 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 43499C0098 for ; Thu, 16 Jul 2015 14:26:55 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.88 X-Spam-Level: ** X-Spam-Status: No, score=2.88 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id Vg_Z9pxUb9hT for ; Thu, 16 Jul 2015 14:26:47 +0000 (UTC) Received: from mail-ig0-f178.google.com (mail-ig0-f178.google.com [209.85.213.178]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id 9A67820EFB for ; Thu, 16 Jul 2015 14:26:47 +0000 (UTC) Received: by igbij6 with SMTP id ij6so14917172igb.1 for ; Thu, 16 Jul 2015 07:25:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=sGvf+kJCNtMbJZud2nKfUlxIz6zl52GqKGIoD5Fobyg=; b=ggqMXcHpXuPq5ntNmo4ia5JiEVwPnqU7/S2YxvMb4EKELbqDiGqBdu7vrZCOQ6myMp jnP4YcI2k4d1/6sRK1lFspJSfferfSvcsXyC6MEqhCaxpdPtP/19k8wRVDrU2yP8Sj+z jOHSlBITK2YagW4z6HumKGIJt8MXFzcJH3VyNuQyBBnF2xTaF7p1g7cklCyZsYlXwPcK GD1mIeZoSKMtQ/U/p6bP2Td7XBJ4Ifi3ANRThlBBjdQjybgVNFmT+MheCzVzNJB0Mcod y1lfszOJViwNPQF8fKJa5xppQIjREJvCqM0Fhj+U5yvmLMYbqPxh23dKWseDCGN3DClw t5Hg== MIME-Version: 1.0 X-Received: by 10.50.79.129 with SMTP id j1mr4838227igx.32.1437056717024; Thu, 16 Jul 2015 07:25:17 -0700 (PDT) Received: by 10.107.44.80 with HTTP; Thu, 16 Jul 2015 07:25:16 -0700 (PDT) Received: by 10.107.44.80 with HTTP; Thu, 16 Jul 2015 07:25:16 -0700 (PDT) In-Reply-To: References: <559C20EE.6000201@gmail.com> Date: Thu, 16 Jul 2015 07:25:16 -0700 Message-ID: Subject: Re: Failed to find an available server in the list of servers From: Billie Rinaldi To: user@accumulo.apache.org Content-Type: multipart/alternative; boundary=089e011609fc23f6e7051afed780 --089e011609fc23f6e7051afed780 Content-Type: text/plain; charset=UTF-8 That's great! :-) On Jul 16, 2015 10:19 AM, "pundu tech" wrote: > Billie, > **THANKS** > Hard to believe I missed this. I was able to login to the shell. > > > On Thu, Jul 16, 2015 at 9:34 AM, Billie Rinaldi > wrote: > >> Regarding the client.conf file, it seems you are using the Property enums >> (such as INSTANCE_NAME) rather than their associated property names >> (such as instance.name). Your client.conf file should look like: >> >> instance.name=comet >> instance.rpc.ssl.enabled=true >> instance.rpc.ssl.clientAuth=true >> >> and so on. If you're generating the file programmatically, you can get >> those property names using the getKey() method of the Property: >> >> Property.INSTANCE_RPC_SSL_ENABLED.getKey() >> >> On Wed, Jul 15, 2015 at 8:05 PM, pundu tech wrote: >> >>> Josh, >>> I had miss this email from you before. >>> >>> So I have done as you suggested. Let me summarize what I have done. >>> >>> 1- Followed >>> https://blogs.apache.org/accumulo/entry/generating_keystores_for_configuring_accumulo >>> I have a master (master)node and 4 slaves (slave1, slave2, slave3, >>> slave4) >>> I have created certificates for the 5 nodes and I have also created >>> certificate for a client which is sitting in slave1. >>> 2-Since I am running the shell from slave1 I have created a client.conf >>> file which I pass to the shell via the --config-file parameter. >>> >>> INSTANCE_NAME=comet >>> >>> INSTANCE_RPC_SSL_ENABLED=true >>> >>> INSTANCE_RPC_SSL_CLIENT_AUTH=true >>> >>> INSTANCE_ZK_HOST=slave1,slave2,slave3,slave4 >>> >>> #the trustore is the same along all the nodes since it stores the pub >>> key of the CA >>> >>> >>> RPC_SSL_TRUSTSTORE_PATH=/home/hadoop/accumulo-1.7.0/conf/clientSSL/truststore.jks >>> >>> RPC_SSL_TRUSTSTORE_TYPE=JKS >>> >>> RPC_SSL_TRUSTSTORE_PASSWORD=accumuloAuth >>> >>> >>> RPC_SSL_KEYSTORE_PATH=/home/hadoop/accumulo-1.7.0/conf/clientSSL/client.jks >>> >>> RPC_SSL_KEYSTORE_TYPE=JKS >>> >>> RPC_SSL_KEYSTORE_PASSWORD=mypass >>> >>> 3-I run the shell with --debug and this is what I get: >>> >>> 2015-07-15 22:53:06,380 [impl.ThriftTransportPool] DEBUG: Failed to >>> connect to ssl:slave1:9997 (120000) >>> >>> org.apache.thrift.transport.TTransportException: Error creating the >>> transport >>> >>> at >>> org.apache.accumulo.core.rpc.ThriftUtil.createSSLContext(ThriftUtil.java:371) >>> >>> at >>> org.apache.accumulo.core.rpc.ThriftUtil.createClientTransport(ThriftUtil.java:248) >>> >>> at >>> org.apache.accumulo.core.client.impl.ThriftTransportPool.createNewTransport(ThriftTransportPool.java:478) >>> >>> at >>> org.apache.accumulo.core.client.impl.ThriftTransportPool.getAnyTransport(ThriftTransportPool.java:466) >>> >>> at >>> org.apache.accumulo.core.client.impl.ServerClient.getConnection(ServerClient.java:141) >>> >>> at >>> org.apache.accumulo.core.client.impl.ServerClient.getConnection(ServerClient.java:117) >>> >>> at >>> org.apache.accumulo.core.client.impl.ServerClient.getConnection(ServerClient.java:113) >>> >>> at >>> org.apache.accumulo.core.client.impl.ServerClient.executeRaw(ServerClient.java:95) >>> >>> at >>> org.apache.accumulo.core.client.impl.ServerClient.execute(ServerClient.java:61) >>> >>> at >>> org.apache.accumulo.core.client.impl.ConnectorImpl.(ConnectorImpl.java:67) >>> >>> at >>> org.apache.accumulo.core.client.ZooKeeperInstance.getConnector(ZooKeeperInstance.java:248) >>> >>> at org.apache.accumulo.shell.Shell.config(Shell.java:362) >>> >>> at org.apache.accumulo.shell.Shell.execute(Shell.java:571) >>> >>> at org.apache.accumulo.start.Main$1.run(Main.java:93) >>> >>> at java.lang.Thread.run(Thread.java:745) >>> >>> Caused by: java.io.IOException: Keystore was tampered with, or password >>> was incorrect >>> >>> at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:772) >>> >>> at >>> sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:55) >>> >>> at java.security.KeyStore.load(KeyStore.java:1214) >>> >>> at >>> org.apache.accumulo.core.rpc.ThriftUtil.createSSLContext(ThriftUtil.java:348) >>> >>> ... 14 more >>> >>> Caused by: java.security.UnrecoverableKeyException: Password >>> verification failed >>> >>> at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:770) >>> >>> This error repeats for every slave. >>> I have tested the password for every keystore and truststore file in the >>> cluster and it is correct--it is the same everywhere. I am very positive >>> about this at this point. Do you have any suggestion on what else could be >>> wrong? >>> >>> I appreciate your help. I am stuck! >>> >>> pundutech >>> >>> On Tue, Jul 7, 2015 at 2:56 PM, Josh Elser wrote: >>> >>>> Pundu, >>>> >>>> The password to use would be the "root" user's password that you set >>>> when calling `accumulo init`. A limitation of the SSL approach is that it >>>> only uses sets up a secure RPC, it isn't a "complete" security >>>> implementation (as you might get with Kerberos in 1.7). >>>> >>>> Sadly, the error messages for SSL are very sparse when the client fails >>>> to negotiate the handshake with a server. With the Accumulo shell, you can >>>> try passing in the --debug option to get more information. >>>> >>>> Alternatively, try turning up org.apache.accumulo.core.client to DEBUG >>>> or TRACE in $ACCUMULO_CONF_DIR/log4j.properties. >>>> >>>> - Josh >>>> >>>> >>>> pundu tech wrote: >>>> >>>>> I have a SSL enabled-accumulo setup. >>>>> >>>>> I have followed: >>>>> >>>>> https://blogs.apache.org/accumulo/entry/generating_keystores_for_configuring_accumulo >>>>> to the teeth and as far as my undersatnding goes on SSL it is all >>>>> correct. >>>>> >>>>> I have created a $ACCUMULO_HOME/conf/client.conf with the following >>>>> properties >>>>> >>>>> INSTANCE_NAME=accumulo >>>>> >>>>> INSTANCE_RPC_SSL_ENABLED=true >>>>> >>>>> NSTANCE_RPC_SSL_CLIENT_AUTH=true >>>>> >>>>> INSTANCE_ZK_HOST=host1 >>>>> >>>>> RPC_SSL_TRUSTSTORE_PATH=/home/hadoop/truststore.jks >>>>> >>>>> RPC_SSL_TRUSTSTORE_TYPE=JKS >>>>> >>>>> RPC_SSL_TRUSTSTORE_PASSWORD=mypass >>>>> >>>>> RPC_SSL_KEYSTORE_PATH=/home/hadoop/server.jks >>>>> >>>>> RPC_SSL_KEYSTORE_TYPE=JKS >>>>> >>>>> RPC_SSL_KEYSTORE_PASSWORD=mypass >>>>> >>>>> >>>>> but when I try to connect via shell I am prompted for a password. Which >>>>> password is this? It does not seem to be the tracer password (which >>>>> user >>>>> is "root"). >>>>> >>>>> ./accumulo shell -u root >>>>> >>>>> /usr/local/zookeeper-3.4.6 >>>>> >>>>> /usr/local/jdk1.7.0_79 >>>>> >>>>> Password: ----> ? >>>>> >>>>> >>>>> Thanks >>>>> >>>>> pundu tech >>>>> >>>>> >>> >> > --089e011609fc23f6e7051afed780 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

That's great! :-)

On Jul 16, 2015 10:19 AM, "pundu tech"= <pundutech@gmail.com> wro= te:
Billie,
**THANKS**
Hard to believe I missed this. I was able t= o login to the shell.=C2=A0


On Thu, Jul 16, 2015 at 9:34 AM,= Billie Rinaldi <billie.rinaldi@gmail.com> wrote:
=
Regarding the cli= ent.conf file, it seems you are using the Property enums (such as INS= TANCE_NAME) rather than their associated property names (such as instance.name).=C2=A0 Y= our client.conf file should look like:

instance.name=3Dcomet
instance.rpc.ssl.e= nabled=3Dtrue
instance.rpc.ssl.clientAuth=3Dtrue

and so on.= =C2=A0 If you're generating the file programmatically, you can get thos= e property names using the getKey() method of the Property:

Property= .INSTANCE_RPC_SSL_ENABLED.getKey()

On Wed, Jul 15, 2015 at 8:05 PM, pundu= tech <pundutech@gmail.com> wrote:
Josh,=C2=A0
I had miss this email from y= ou before.

So I have done as you suggested.=C2=A0 = Let me summarize what I have done.

I have a master (master= )node and 4 slaves (slave1, slave2, slave3, slave4)
I have create= d certificates for the 5 nodes and I have also created certificate for a cl= ient which is sitting in slave1.
2-Since I am running the shell f= rom slave1 I have created a client.conf file which I pass to the shell via = the --config-file parameter.

INSTANCE_NAME=3Dcomet

INSTANCE_RPC_SSL_ENABLED=3Dtrue=

INSTANCE_RPC_SSL_CLIENT_AUTH=3Dtrue

INSTANCE_ZK_HOST=3Dslave1,slave2,slave3,slave4

#the trustore is the same along all the nodes since it stores the = pub key of the CA

RPC_SSL_TRUSTSTORE_PATH=3D/home/hadoop= /accumulo-1.7.0/conf/clientSSL/truststore.jks

RPC_SSL_TRUSTSTORE_TYPE=3DJKS

RPC_SSL_TRUSTSTORE_PASSWORD=3DaccumuloAuth

RPC_SSL_KEYSTORE_PATH=3D/home/hadoop/accumulo-1.7.0/conf/clientSSL= /client.jks

RPC_SSL_KEYSTORE_TYPE=3DJKS

RPC_SSL_KEYSTORE_PASSWORD=3Dmypass

3-I run the = shell with --debug and this is what I get:

2015-07-15 22= :53:06,380 [impl.ThriftTransportPool] DEBUG: Failed to connect to ssl:slave= 1:9997 (120000)

org.apache.thrift.transport.TTransportEx= ception: Error creating the transport

at o= rg.apache.accumulo.core.rpc.ThriftUtil.createSSLContext(ThriftUtil.java:371= )

at org.apache.accumulo.core.rpc.ThriftUt= il.createClientTransport(ThriftUtil.java:248)

at org.apache.accumulo.core.client.impl.ThriftTransportPool.createNewTr= ansport(ThriftTransportPool.java:478)

at o= rg.apache.accumulo.core.client.impl.ThriftTransportPool.getAnyTransport(Thr= iftTransportPool.java:466)

at org.apache.a= ccumulo.core.client.impl.ServerClient.getConnection(ServerClient.java:141)<= /span>

at org.apache.accumulo.core.client.impl.Se= rverClient.getConnection(ServerClient.java:117)

<= /span>at org.apache.accumulo.core.client.impl.ServerClient.getConnection(Se= rverClient.java:113)

at org.apache.accumul= o.core.client.impl.ServerClient.executeRaw(ServerClient.java:95)

=

at org.apache.accumulo.core.client.impl.ServerClient= .execute(ServerClient.java:61)

at org.apac= he.accumulo.core.client.impl.ConnectorImpl.<init>(ConnectorImpl.java:= 67)

at org.apache.accumulo.core.client.Zoo= KeeperInstance.getConnector(ZooKeeperInstance.java:248)

= at org.apache.accumulo.shell.Shell.config(Shell.java:362)

at org.apache.accumulo.shell.Shell.execute(Sh= ell.java:571)

at org.apache.accumulo.start= .Main$1.run(Main.java:93)

at java.lang.Thr= ead.run(Thread.java:745)

Caused by: java.io.IOException:= Keystore was tampered with, or password was incorrect

<= span> at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.= java:772)

at sun.security.provider.JavaKey= Store$JKS.engineLoad(JavaKeyStore.java:55)

at java.security.KeyStore.load(KeyStore.java:1214)

at org.apache.accumulo.core.rpc.ThriftUtil.createSSLContext(Thrif= tUtil.java:348)

... 14 more

<= span>Caused by: java.security.UnrecoverableKeyException: Password verificat= ion failed

at sun.security.provider.JavaKeyStore.eng= ineLoad(JavaKeyStore.java:770)


This err= or repeats for every slave.
I have tested the password for every = keystore and truststore file in the cluster and it is correct--it is the sa= me everywhere.=C2=A0 I am very positive about this at this point.=C2=A0 Do = you have any suggestion on what else could be wrong?

I appreciate your help. I am stuck!

pundutech

On Tue, Jul 7, 2015 at 2:56 PM, Josh Elser <josh.elser@gmail.com= > wrote:
Pundu,

The password to use would be the "root" user's password that = you set when calling `accumulo init`. A limitation of the SSL approach is t= hat it only uses sets up a secure RPC, it isn't a "complete" = security implementation (as you might get with Kerberos in 1.7).

Sadly, the error messages for SSL are very sparse when the client fails to = negotiate the handshake with a server. With the Accumulo shell, you can try= passing in the --debug option to get more information.

Alternatively, try turning up org.apache.accumulo.core.client to DEBUG or T= RACE in $ACCUMULO_CONF_DIR/log4j.properties.<= br>
- Josh


pundu tech wrote:
I have a SSL enabled-accumulo setup.

I have followed:
https://blogs.a= pache.org/accumulo/entry/generating_keystores_for_configuring_accumulo<= br> =C2=A0 to the teeth and as far as my undersatnding goes on SSL it is all co= rrect.

I have created a $ACCUMULO_HOME/conf/client.conf with the following
properties

INSTANCE_NAME=3Daccumulo

INSTANCE_RPC_SSL_ENABLED=3Dtrue

NSTANCE_RPC_SSL_CLIENT_AUTH=3Dtrue

INSTANCE_ZK_HOST=3Dhost1

RPC_SSL_TRUSTSTORE_PATH=3D/home/hadoop/truststore.jks

RPC_SSL_TRUSTSTORE_TYPE=3DJKS

RPC_SSL_TRUSTSTORE_PASSWORD=3Dmypass

RPC_SSL_KEYSTORE_PATH=3D/home/hadoop/server.jks

RPC_SSL_KEYSTORE_TYPE=3DJKS

RPC_SSL_KEYSTORE_PASSWORD=3Dmypass


but when I try to connect via shell I am prompted for a password. Which
password is this? It does not seem to be the tracer password (which user is "root").

./accumulo shell -u root

/usr/local/zookeeper-3.4.6

/usr/local/jdk1.7.0_79

Password:=C2=A0 =C2=A0----> ?


Thanks

pundu tech




--089e011609fc23f6e7051afed780--