hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Dimiduk <ndimi...@gmail.com>
Subject Re: Verification of Short Circuit Reads
Date Fri, 04 Nov 2016 22:07:50 GMT
That closing question should have been "so I add the _hbase_ user to the
hdfs group?"

On Thursday, November 3, 2016, Nick Dimiduk <ndimiduk@gmail.com> wrote:

> Hello there,
>
> I'm setting up a new cluster and I notice in my RS startup logs an ominous
> warning.
>
> 2016-11-03 21:52:14,624 WARN  [RS_LOG_REPLAY_OPS-r103u3:16020-0]
> shortcircuit.DomainSocketFactory: error creating DomainSocket
> java.net.ConnectException: connect(2) error: Permission denied when trying
> to connect to '/var/run/hadoop-hdfs/dn.socket'
>
> My hdfs-site.xml (not not hbase-site.xml) has:
>
>   <property>
>     <name>dfs.client.read.shortcircuit</name>
>     <value>true</value>
>   </property>
>   <property>
>     <name>dfs.client.read.shortcircuit.streams.cache.size</name>
>     <value>4096</value>
>   </property>
>   <property>
>     <name>dfs.domain.socket.path</name>
>     <value>/var/run/hadoop-hdfs/dn.socket</value>
>   </property>
>
> lsof on the RS process shows me that libhadoop.so has been loaded.
>
> $ sudo lsof -p $(cat /var/run/hbase/hbase-hbase-regionserver.pid) | grep
> 'libhadoop\.so'
> java    33581 hbase  mem    REG              252,0   137720  2492109
> /usr/hdp/2.3.2.0-2950/hadoop/lib/native/libhadoop.so.1.0.0
>
> Given the hbase user's group membership,
>
> $ sudo -u hbase groups
> hbase hadoop
>
> Permission on the socket itself looks suspicious
>
> $ sudo ls -la /var/run/hadoop-hdfs
> total 0
> drwx------  2 hdfs root  60 Nov  2 22:56 .
> drwxr-xr-x 23 root root 760 Nov  3 21:52 ..
> srw-rw-rw-  1 hdfs hdfs   0 Nov  2 22:56 dn.socket
>
> Our book has this word of caution: "Be careful about permissions for the
> directory that hosts the shared domain socket; dfsclient will complain if
> open to other than the hbase user." But this seems inaccurate in that I'm
> seeing HDFS complain if the directory is open to a user other than hdfs.
>
> So what's the correct solution here? Do I add hdfs user to the hdfs group?
> That sounds too permissive.
>
> Thanks,
> Nick
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message