accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geoffry Roberts <threadedb...@gmail.com>
Subject Re: Connecting java client to Accumulo VM
Date Thu, 09 Nov 2017 18:23:30 GMT
Thanks for the best practice advice wrt accumulo-env.sh.  I was using
.profile because Hadoop finds it well enough.


On Thu, Nov 9, 2017 at 12:04 PM, Josh Elser <elserj@apache.org> wrote:

> +1 to Christopher's assessment of the JAVA_HOME issue.
>
> A bit more color if you find it interesting: Accumulo tends to do a bit of
> SSH'ing in starting up processes (this has gotten better in the 1.8 or 2.0
> scripts). As such, it's a best-practice to explicitly set environment
> variables you need in accumulo-env.sh.
>
> On 11/9/17 11:52 AM, Christopher wrote:
>
>> So, Accumulo TServers publish their hostname from their config into the
>> Accumulo metadata. So, yes, this hostname must be reachable by clients as
>> well, because that's how clients will identify it in the Accumulo metadata.
>>
>> I'm not sure what the deal was with the JAVA_HOME variable. It's probably
>> one of the launch scripts which ssh'd to the hostname in the config to
>> start the tserver. When it's set to localhost, it probably doesn't ssh, but
>> just starts it directly. My guess is that .profile is not read when ssh'ing
>> because it's not creating a logon session and instead just executing the
>> given command. You could try putting it in .bashrc instead (assuming your
>> default shell is bash).
>>
>> On Thu, Nov 9, 2017 at 11:42 AM Geoffry Roberts <threadedblue@gmail.com
>> <mailto:threadedblue@gmail.com>> wrote:
>>
>>     All,
>>
>>     Muchas gracias for the help.
>>
>>     When I had masters/slaves populated with "localhost", 9997 was
>>     listening but not remotely.  When I had m/s populated with the IP
>>     address,  9997 was not listening and the log said JAVA_HOME was not
>>     set and that I should set it in accumulo-env.sh.  I did that and
>>     9997 began listening remotely and was accessible.  Let us then,
>>     declare victory.
>>
>>     Two points, however, I should like to pass along:
>>
>>      1. Why the problem with JAVA_HOME?  I have JAVA_HOME set in
>>         .profile.  When m/s is "localhost" no problem; but when
>>         "hostname" (from /etc/hosts), big problem.
>>      2. Before I could get my remote client to connect, I had to make a
>>         mirror entry in the host's /etc/hosts file.  i.e. Both host and
>>         guest machines needed the same entry.  This is because the
>>         client tries to work with the name from the guest's /etc/hosts
>>         and not the IP.
>>     Anyhow, I am off to the races with Accumulo.
>>
>>     On Wed, Nov 8, 2017 at 5:49 PM, Edward Gleeck <edrebg@gmail.com
>>     <mailto:edrebg@gmail.com>> wrote:
>>
>>         Yep should work. would suggest checking the logs at this point
>>         to see  what’s causing the failure.  If it’s not starting up
>>         there would be exceptions thrown by the service.
>>
>>         On Wed, Nov 8, 2017 at 5:36 PM Geoffry Roberts
>>         <threadedblue@gmail.com <mailto:threadedblue@gmail.com>> wrote:
>>
>>             I tried the IP address (a 192 number) but the same
>>             result--no 9997.  Using said IP I can access from either the
>>             host or from within the guest.
>>
>>             So far nothing works in master/slaves except localhost.
>>
>>             I gather this is supposed to work correct?
>>
>>             On Wed, Nov 8, 2017 at 5:16 PM, Edward Gleeck
>>             <edrebg@gmail.com <mailto:edrebg@gmail.com>> wrote:
>>
>>                 You wouldn't want the 0.0.0.0 on your /etc/hosts as this
>>                 wouldn't be valid. I don't recall exactly which
>>                 configuration file under $ACCUMULO_CONF_DIR you would
>>                 want this in as Josh pointed out, but if you were to go
>>                 the /etc/host route, you want to put the IP address of
>>                 that interface VM host. for example /etc/hosts:
>>
>>                 192.168.56.101 localhost localhost.localdomain
>>
>>                 HTH
>>
>>
>>
>>                 On Wed, Nov 8, 2017 at 4:13 PM, Geoffry Roberts
>>                 <threadedblue@gmail.com <mailto:threadedblue@gmail.com>>
>>                 wrote:
>>
>>                     I gave your suggestion a try.  I made an entry in
>>                     /etc/hosts that resolves to 0.0.0.0 then set that
>>                     name in master and slaves.  (I am running single
>>                     node for now.). The upshot is port 9997 does not
>>                     appear as listening at all.  If I change back to
>>                     localhost, then it appears again.   My guess is the
>>                     tablet server only starts when it's port is localhost.
>>
>>                     Am I using Accumulo correctly?  Is it not designed
>>                     to be accessed remotely?
>>
>>
>>                     On Wed, Nov 8, 2017 at 2:20 PM, Josh Elser
>>                     <josh.elser@gmail.com <mailto:josh.elser@gmail.com>>
>>
>>                     wrote:
>>
>>                         Accumulo chooses the network interface to bind
>>                         given the resolution of the hostname that you
>>                         provide in the "hosts" files in ACCUMULO_CONF_DIR.
>>
>>                         If you have "localhost" (the default) still in
>>                         the files (e.g. masters, slaves), this
>>                         presumably resolves to 127.0.0.1 which will
>>                         result in Accumulo not accepting connections
>>                         from your VM's network adapter.
>>
>>                         A quick hack would be to put "0.0.0.0" in those
>>                         files instead of "localhost". I think the
>>                         Accumulo scripts only have the ability to
>>                         override the bound interface for the Monitor,
>>                         not all processes, to be 0.0.0.0. You could also
>>                         use a hostname you define in /etc/hosts that
>>                         binds to the proper interface instead (which
>>                         would be a bit more like reality).
>>
>>                         On 11/8/17 10:43 AM, Geoffry Roberts wrote:
>>
>>                             All,
>>
>>                             I have used Accumulo before, but a few
>>                             versions ago (1.5.1), maybe something has
>>                             changed.  Also, I've never before run it in
>>                             a VM.
>>
>>                             I am running Accumulo from withn a VM and
>>                             attempting to connect from without.  I am
>>                             getting a complaint regarding port 9997,
>>                             which, within the VM, is listening on
>>                             127.0.0.1:9997 <http://127.0.0.1:9997>
>>                             <http://127.0.0.1:9997>.  Apparently, I need
>>                             to get it onto 0.0.0.0:9997
>>                             <http://0.0.0.0:9997>
>>                             <http://0.0.0.0:9997>.  Am I correct?
>>
>>                             Hadoop 2.6.2
>>                             Zookeeper 3.4.10
>>                             Accumulo 1.8.1
>>                             Thrift 0.10.0
>>                             Ubuntu 16.04 as a VBox guest
>>                             OSX 10.12.06 as the host
>>
>>                             The HAZ is running on the VM.  From the CLI
>>                             I can connect with the Zookeeper client and
>>                             the Accumulo client.
>>
>>                             I have ports 22, 50070, 50075, 2181, 9995,
>>                             and 9997 forwarded.
>>
>>                               From the host, I can browse the Hadoop and
>>                             Accumulo web pages.  I can also connect
>>                             using the Zookeeper client.  When I try to
>>                             connect from Java,  I am getting a
>>                             successful connect to 2181, but it complains
>>                             about port 9997.
>>
>>                             The VM is showing all forwarded ports as
>>                             listening on 0.0.0.0 <http://0.0.0.0>:<port>
>>                             except for 127.0.0.1:9997
>>                             <http://127.0.0.1:9997>
>>                             <http://127.0.0.1:9997>.  As best I can tell
>>                             Thrift needs the port.
>>
>>                             My connect code:
>>
>>                             String instance=
>>                             properties.getProperty(ACCUMULO_INSTANCE);
>>
>>                             String uri =
>>                             properties.getProperty(ZOOKEEPER_URI);
>>
>>                             Instance inst = new
>>                             ZooKeeperInstance(instance, uri);
>>
>>                             String user = properties.getProperty(USER);
>>
>>                             String password =
>>                             properties.getProperty(PASSWORD);
>>
>>                             AuthenticationToken token = new
>>                             PasswordToken(password);
>>
>>                             conn = inst.getConnector(user, token);
>>
>>
>>                             Errors from Java:
>>
>>                             2017-11-08 09:50:55 INFO  ZooKeeper:438 -
>>                             Initiating client connection,
>>                             connectString=192.168.50.101:2181
>>                             <http://192.168.50.101:2181>
>>                             <http://192.168.50.101:2181>
>>                             sessionTimeout=30000
>>                             watcher=org.apache.accumulo.fa
>> te.zookeeper.ZooSession$ZooWatcher@4b013c76
>>
>>                             2017-11-08 09:50:55 INFO  ClientCnxn:1032 -
>>                             Opening socket connection to server
>>                             192.168.50.101/192.168.50.101:2181
>>                             <http://192.168.50.101/192.168.50.101:2181>
>>                             <http://192.168.50.101/192.168.50.101:2181>.
>>                             Will not attempt to authenticate using SASL
>>                             (unknown error)
>>
>>                             2017-11-08 09:50:55 INFO  ClientCnxn:876 -
>>                             Socket connection established to
>>                             192.168.50.101/192.168.50.101:2181
>>                             <http://192.168.50.101/192.168.50.101:2181>
>>                             <http://192.168.50.101/192.168.50.101:2181>,
>>                             initiating session
>>
>>                             2017-11-08 09:50:55 INFO  ClientCnxn:1299 -
>>                             Session establishment complete on server
>>                             192.168.50.101/192.168.50.101:2181
>>                             <http://192.168.50.101/192.168.50.101:2181>
>>                             <http://192.168.50.101/192.168.50.101:2181>,
>>                             sessionid = 0x15f9c185f8e0006, negotiated
>>                             timeout = 30000
>>
>>                             2017-11-08 09:50:58 WARN
>>        ThriftTransportPool:539 - Server
>>                             localhost:9997 (120000) had 20 failures in a
>>                             short time period, will not complain anymore
>>
>>                             ...
>>                             It then continues to try to connect to no
>> avail.
>>
>>
>>                             Thanks in advance
>>
>>                             --                             There are ways
>> and there are ways,
>>
>>                             Geoffry Roberts
>>
>>
>>
>>
>>                     --                     There are ways and there are
>> ways,
>>
>>                     Geoffry Roberts
>>
>>
>>
>>
>>
>>             --             There are ways and there are ways,
>>
>>             Geoffry Roberts
>>
>>         --         Sent from handheld device
>>
>>
>>
>>
>>     --     There are ways and there are ways,
>>
>>     Geoffry Roberts
>>
>>


-- 
There are ways and there are ways,

Geoffry Roberts

Mime
View raw message