accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Miller (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-4640) Accumulo shell is expecting instance.volumes in client.conf instead of accumulo-site.xml
Date Wed, 31 May 2017 14:49:04 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-4640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16031267#comment-16031267
] 

Michael Miller commented on ACCUMULO-4640:
------------------------------------------

bq.  Essentially with "-z" (or "-zi") you are giving the shell the information that it would
otherwise have to figure out from HDFS after looking up the value of instance.volumes in accumulo-site.xml
(or client.conf since 1.8.1).

Correct, except the Shell first looks for "instance.name" and "instance.zookeeper.host" from
client.conf.  If it can't find either of those two properties, it will than try to read them
from accumulo-site.xml.  The "instance.volumes" property is not a client property, it is only
for use in the server configuration file, accumulo-site.xml

bq. I guess the question is then: when "-z" (or "-zi") are not provided as arguments to it
should the shell pick up the value of instance.volumes from accumulo-site.xml or client.conf?
1.8.1 takes the value from client.conf where 1.7.x and 1.8.0 take the value from accumulo-site.xml.

Your client.conf should set the "instance.name" and "instance.zookeeper.host" properties.
 This will prevent the shell from having to read accumulo-site.xml.

I can see how this appears to be a bug but this change was made so the Shell would function
as intended (more like a client).  Previously it was always reading from accumulo-site.xml.
 The Shell was only supposed to fall back to reading from accumulo-site.xml (for backwards
compatibility) when it couldn't get the information it needed from client.cont.  Now, with
this change you can properly secure accumulo-site.xml and run the shell with "-z" (or "-zi"
and "-zh").  

> Accumulo shell is expecting instance.volumes in client.conf instead of accumulo-site.xml
> ----------------------------------------------------------------------------------------
>
>                 Key: ACCUMULO-4640
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4640
>             Project: Accumulo
>          Issue Type: Bug
>          Components: shell
>    Affects Versions: 1.8.1
>         Environment: [accumulo@master ~]$ cat /etc/redhat-release
> CentOS Linux release 7.3.1611 (Core)
> [accumulo@master ~]$ uname -a
> Linux master 3.10.0-514.10.2.el7.x86_64 #1 SMP Fri Mar 3 00:04:05 UTC 2017 x86_64 x86_64
x86_64 GNU/Linux
> [accumulo@master ~]$ java -version
> java version "1.8.0_112"
> Java(TM) SE Runtime Environment (build 1.8.0_112-b15)
> Java HotSpot(TM) 64-Bit Server VM (build 25.112-b15, mixed mode)
> Hadoop: 2.8.0
> ZooKeeper: 3.4.10
> Accumulo: 1.8.1
>            Reporter: Luigi Di Fraia
>             Fix For: 1.8.2, 2.0.0
>
>         Attachments: accumulo-env.sh, accumulo-site.xml
>
>
> Starting with Accumulo 1.8.1 we are getting the exception as per below when the Accumulo
shell is invoked:
> [accumulo@master ~]$ /usr/local/accumulo/bin/accumulo shell -u root -p ***removed***
> 2017-05-18 14:31:23,667 [zookeeper.ZooUtil] ERROR: unable obtain instance id at hdfs://master:9000/accumulo/instance_id
> 2017-05-18 14:31:23,669 [start.Main] ERROR: Thread 'shell' died.
> java.lang.RuntimeException: Accumulo not initialized, there is no instance id at hdfs://master:9000/accumulo/instance_id
>         at org.apache.accumulo.core.zookeeper.ZooUtil.getInstanceIDFromHdfs(ZooUtil.java:66)
>         at org.apache.accumulo.core.zookeeper.ZooUtil.getInstanceIDFromHdfs(ZooUtil.java:51)
>         at org.apache.accumulo.shell.Shell.getZooInstance(Shell.java:511)
>         at org.apache.accumulo.shell.Shell.setInstance(Shell.java:473)
>         at org.apache.accumulo.shell.Shell.config(Shell.java:322)
>         at org.apache.accumulo.shell.Shell.execute(Shell.java:589)
>         at org.apache.accumulo.start.Main$1.run(Main.java:120)
>         at java.lang.Thread.run(Thread.java:745)
> The location used to look up the instance_id (hdfs://master:9000/accumulo/instance_id)
does not appear to be consistent with this site configuration and I am not quite sure how
it is being worked out.
> The Accumulo Overview web page shows that the instance exists:
> Instance ID: 4264f11a-704f-4c1f-880f-08f6154232b2 
> Thu May 18 14:44:03 BST 2017
> The instance_id file is in Hadoop's HDFS:
> [accumulo@master ~]$ /usr/local/hadoop/bin/hdfs dfs -ls /user/accumulo/accumulo/instance_id
> Found 1 items
> -rw-r--r--   3 accumulo supergroup          0 2017-05-18 14:27 /user/accumulo/accumulo/instance_id/4264f11a-704f-4c1f-880f-08f6154232b2
> Again, within the same environment as selected above apart for using Accumulo 1.8.0,
the shell can be invoked without issues.
> Here's a snippet from accumulo-site.xml:
>   <property>
>     <name>instance.volumes</name>
>     <value>hdfs://master:9000/user/accumulo/accumulo</value>
>     <description>comma separated list of URIs for volumes. example: hdfs://localhost:9000/accumulo</description>
>   </property>
> For historical reasons our "root volume" is not / but /user/accumulo although the above
stack trace suggests that / is being used as root volume.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message