hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: Verification of Short Circuit Reads
Date Sun, 06 Nov 2016 20:43:01 GMT
Did some quick searches out of curiosity which state that unix 
filesystem permissions should be sufficient (the hbase user would not 
need to be in the hdfs group).

Is the permission on /var/run/hadoop-hdfs set correctly? (hbase user 
could do that same `ls`)

Nick Dimiduk wrote:
> That closing question should have been "so I add the _hbase_ user to the
> hdfs group?"
>
> On Thursday, November 3, 2016, Nick Dimiduk<ndimiduk@gmail.com>  wrote:
>
>> Hello there,
>>
>> I'm setting up a new cluster and I notice in my RS startup logs an ominous
>> warning.
>>
>> 2016-11-03 21:52:14,624 WARN  [RS_LOG_REPLAY_OPS-r103u3:16020-0]
>> shortcircuit.DomainSocketFactory: error creating DomainSocket
>> java.net.ConnectException: connect(2) error: Permission denied when trying
>> to connect to '/var/run/hadoop-hdfs/dn.socket'
>>
>> My hdfs-site.xml (not not hbase-site.xml) has:
>>
>>    <property>
>>      <name>dfs.client.read.shortcircuit</name>
>>      <value>true</value>
>>    </property>
>>    <property>
>>      <name>dfs.client.read.shortcircuit.streams.cache.size</name>
>>      <value>4096</value>
>>    </property>
>>    <property>
>>      <name>dfs.domain.socket.path</name>
>>      <value>/var/run/hadoop-hdfs/dn.socket</value>
>>    </property>
>>
>> lsof on the RS process shows me that libhadoop.so has been loaded.
>>
>> $ sudo lsof -p $(cat /var/run/hbase/hbase-hbase-regionserver.pid) | grep
>> 'libhadoop\.so'
>> java    33581 hbase  mem    REG              252,0   137720  2492109
>> /usr/hdp/2.3.2.0-2950/hadoop/lib/native/libhadoop.so.1.0.0
>>
>> Given the hbase user's group membership,
>>
>> $ sudo -u hbase groups
>> hbase hadoop
>>
>> Permission on the socket itself looks suspicious
>>
>> $ sudo ls -la /var/run/hadoop-hdfs
>> total 0
>> drwx------  2 hdfs root  60 Nov  2 22:56 .
>> drwxr-xr-x 23 root root 760 Nov  3 21:52 ..
>> srw-rw-rw-  1 hdfs hdfs   0 Nov  2 22:56 dn.socket
>>
>> Our book has this word of caution: "Be careful about permissions for the
>> directory that hosts the shared domain socket; dfsclient will complain if
>> open to other than the hbase user." But this seems inaccurate in that I'm
>> seeing HDFS complain if the directory is open to a user other than hdfs.
>>
>> So what's the correct solution here? Do I add hdfs user to the hdfs group?
>> That sounds too permissive.
>>
>> Thanks,
>> Nick
>>
>

Mime
View raw message