hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Wolf <opus...@gmail.com>
Subject Re: EC2 remote client woes
Date Mon, 16 Jan 2012 17:44:36 GMT
Many, many thanks J-D

I grepped 'hbase classpath' and found this...

    /usr/lib/hbase/bin/../conf

/usr/lib/hbase/conf contains a hbase-site.xml which is identical to the 
one in /etc/hbase/conf.  In fact /usr/lib/hbase/conf is just a link to 
/etc/hbase/conf.

So, I think it should have worked.  Furthermore, it seems to work on my 
Mac OSX (Lion).

Here is a new pastbin, with a log of 'hbase shell -d'

     http://pastebin.com/78zJkdti

Looking at commands.rb, it seems to be just allocating an HBaseAdmin.  
Is there some additional logging I can switch on to help isolate this?

P





On 1/13/12 9:12 PM, Jean-Daniel Cryans wrote:
> Apache HBase doesn't require that, as you can tell by the
> documentation: http://hbase.apache.org/book.html
>
> You are running CDH3u2, which has its own quirks/features, so as to
> why it's not picking it up while it should is a question for the
> cloudera mailing lists.
>
> J-D
>
> On Fri, Jan 13, 2012 at 5:24 PM, Peter Wolf<opus111@gmail.com>  wrote:
>> My apologies,
>>
>> Here is what "hbase classpath" returns
>>
>> http://pastebin.com/2MP9c6Yq
>>
>> As you can see, /etc/hbase/conf is not on there, so that explains my
>> problem.  The documentation indicates that hbase-site.xml is normally
>> located in /etc/hbase/conf.  Is this a problem with 'hbase', or did I
>> mis-understand?
>>
>> Thanks
>> P
>>
>>
>>
>>
>> On 1/13/12 12:43 PM, Jean-Daniel Cryans wrote:
>>> Sorry what I meant by "pastebin all the debug" was to use a service
>>> like pastebin.com to keep the emails short.
>>>
>>> So in there I see:
>>>
>>>> 12/01/13 02:21:22 INFO zookeeper.ClientCnxn: Opening socket connection to
>>>> server localhost/127.0.0.1:2181
>>> Which means that it's connecting to the default value of
>>> hbase.zookeeper.quorum which is localhost. I don't know how you set
>>> HBase on your remote machine but it's not picking up your
>>> hbase-site.xml. It should normally be in the shell's classpath if you
>>> start it with "bin/hbase shell", but you can easily verify that by
>>> doing "bin/hbase classpath" and you should see which conf directory is
>>> being picked up.
>>>
>>> J-D
>>>
>>> On Thu, Jan 12, 2012 at 6:24 PM, Peter Wolf<opus111@gmail.com>    wrote:
>>>> Here you go, and thanks!
>>>>
>>>> P
>>>>
>>>>
>>>>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>>>>         at
>>>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
>>>>         at
>>>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
>>>> 12/01/13 02:21:20 DEBUG zookeeper.ClientCnxn: Ignoring exception during
>>>> shutdown input
>>>> java.nio.channels.ClosedChannelException
>>>>         at
>>>> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:656)
>>>>         at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:378)
>>>>         at
>>>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1205)
>>>>         at
>>>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1170)
>>>> 12/01/13 02:21:20 DEBUG zookeeper.ClientCnxn: Ignoring exception during
>>>> shutdown output
>>>> java.nio.channels.ClosedChannelException
>>>>         at
>>>> sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:667)
>>>>         at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:386)
>>>>         at
>>>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1212)
>>>>         at
>>>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1170)
>>>> 12/01/13 02:21:20 DEBUG zookeeper.ZooKeeperWatcher: Retrying zk create
>>>> for
>>>> another 8565ms; set 'hbase.zookeeper.recoverable.waittime' to change wait
>>>> time); KeeperErrorCode = ConnectionLoss for /hbase
>>>> 12/01/13 02:21:22 INFO zookeeper.ClientCnxn: Opening socket connection to
>>>> server localhost/127.0.0.1:2181
>>>> 12/01/13 02:21:22 WARN zookeeper.ClientCnxn: Session 0x0 for server null,
>>>> unexpected error, closing socket connection and attempting reconnect
>>>> java.net.ConnectException: Connection refused
>>>>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>>>>         at
>>>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
>>>>         at
>>>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
>>>> 12/01/13 02:21:22 DEBUG zookeeper.ClientCnxn: Ignoring exception during
>>>> shutdown input
>>>> java.nio.channels.ClosedChannelException
>>>>         at
>>>> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:656)
>>>>         at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:378)
>>>>         at
>>>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1205)
>>>>         at
>>>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1170)
>>>> 12/01/13 02:21:22 DEBUG zookeeper.ClientCnxn: Ignoring exception during
>>>> shutdown output
>>>> java.nio.channels.ClosedChannelException
>>>>         at
>>>> sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:667)
>>>>         at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:386)
>>>>         at
>>>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1212)
>>>>         at
>>>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1170)
>>>> 12/01/13 02:21:22 DEBUG zookeeper.ZooKeeperWatcher: Retrying zk create
>>>> for
>>>> another 6603ms; set 'hbase.zookeeper.recoverable.waittime' to change wait
>>>> time); KeeperErrorCode = ConnectionLoss for /hbase
>>>>
>>>>
>>>>
>>>>
>>>> On 1/12/12 5:36 PM, Jean-Daniel Cryans wrote:
>>>>> Interesting, could you start the shell with "-d" and pastebin all the
>>>>> debug that comes out after the first command?
>>>>>
>>>>> BTW the shell does work on remote clusters, so it's some other issue.
>>>>>
>>>>> J-D
>>>>>
>>>>> On Thu, Jan 12, 2012 at 1:56 PM, Peter Wolf<opus111@gmail.com>
     wrote:
>>>>>> Sorry, that's a typo in my email.  Here is my config file again (that
>>>>>> doesn't work)
>>>>>>
>>>>>>
>>>>>> <configuration>
>>>>>> <property>
>>>>>> <name>hbase.zookeeper.quorum</name>
>>>>>> <value>ip-AA-BBB-C-DDD.ec2.internal</value>
>>>>>> <description>Standalone Server</description>
>>>>>> </property>
>>>>>> </configuration>
>>>>>>
>>>>>> I double checked, and I am using ip-AA-BBB-C-DDD.ec2.internal
>>>>>> consistently
>>>>>> in config files and code.
>>>>>>
>>>>>> P
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 1/12/12 4:24 PM, Jean-Daniel Cryans wrote:
>>>>>>> Yes, it's the same thing, which is why I think the additional
>>>>>>> ec2.internal in your hbase-site is suspicious. Let me reiterate:
>>>>>>>
>>>>>>> This works:
>>>>>>>
>>>>>>> echo stat|nc ip-XX-YYY-Z-QQQ.ec2.internal 2181
>>>>>>>
>>>>>>> But this config doesn't:
>>>>>>>
>>>>>>> <value>ip-XX-YYY-Z-QQQ.ec2.internal.ec2.internal</value>
>>>>>>>
>>>>>>> Now what happens if you just use the same one, the one that works?
>>>>>>>
>>>>>>> J-D
>>>>>>>
>>>>>>> On Thu, Jan 12, 2012 at 12:45 PM, Peter Wolf<opus111@gmail.com>
>>>>>>>   wrote:
>>>>>>>> I'm a N00B, so I'm not sure of anything... but it is working
now
>>>>>>>> using
>>>>>>>> the
>>>>>>>> Java Client API, and XXX.ec2.internal address on both server
and
>>>>>>>> client.
>>>>>>>>
>>>>>>>> The problem seems to be 'hbase shell', which is odd as I
would have
>>>>>>>> thought
>>>>>>>> it sat on top of the Java API.
>>>>>>>>
>>>>>>>> P
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 1/12/12 1:22 PM, Jean-Daniel Cryans wrote:
>>>>>>>>> Your config file on the remote machine has:
>>>>>>>>>
>>>>>>>>> ip-XX-YYY-Z-QQQ.ec2.internal.ec2.internal
>>>>>>>>>
>>>>>>>>> You sure about the extra ec2.internal?
>>>>>>>>>
>>>>>>>>> J-D
>>>>>>>>>
>>>>>>>>> On Thu, Jan 12, 2012 at 9:26 AM, Peter Wolf<opus111@gmail.com>
>>>>>>>>>   wrote:
>>>>>>>>>> Oh yeah!  The code did it :-D
>>>>>>>>>>
>>>>>>>>>> For those that come after, I guess 'hbase shell'
is broken for
>>>>>>>>>> remote
>>>>>>>>>> access.  Use the raw Java API
>>>>>>>>>>
>>>>>>>>>> Many thanks again Mark!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 1/12/12 11:40 AM, Mark Kerzner wrote:
>>>>>>>>>>> 1. Look in the logs;
>>>>>>>>>>> 2. I think hbase shell works only locally;
>>>>>>>>>>> 3. The code below worked for me, and I don't
use a config file,
>>>>>>>>>>> but
>>>>>>>>>>> give
>>>>>>>>>>> the params directly:
>>>>>>>>>>>
>>>>>>>>>>>      public void connect() throws IOException
{
>>>>>>>>>>>          Configuration hConf = HBaseConfiguration.create();
>>>>>>>>>>>
>>>>>>>>>>> hConf.set(MyConstants.HBASE_CONFIGURATION_ZOOKEEPER_QUORUM,
>>>>>>>>>>> MyConstants.zookeeperUrl);
>>>>>>>>>>>
>>>>>>>>>>> hConf.set(MyConstants..HBASE_CONFIGURATION_ZOOKEEPER_CLIENTPORT,
>>>>>>>>>>> MyConstants.zookeeperPort);
>>>>>>>>>>>          String tableName = "MyTable";
>>>>>>>>>>>          HTable hTable = new HTable(hConf, tableName);
>>>>>>>>>>>          hTable.close();
>>>>>>>>>>>      }
>>>>>>>>>>>
>>>>>>>>>>> Mark
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jan 12, 2012 at 10:20 AM, Peter Wolf<opus111@gmail.com>
>>>>>>>>>>>   wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Still no love...  Any suggestions?
>>>>>>>>>>>>
>>>>>>>>>>>> I'm on EC2, and I am trying to set up a Pseudo-Distributed
HBaser
>>>>>>>>>>>> Server
>>>>>>>>>>>> on one machine, and access it from another.
 Both machines are
>>>>>>>>>>>> EC2.
>>>>>>>>>>>>
>>>>>>>>>>>> I have already found the doc below, and I
followed the
>>>>>>>>>>>> instructions
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> http://hbase.apache.org/book.**html#client_dependencies<http://hbase.apache.org/book.html#client_dependencies>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> My server is up and running, and I can access
it from an 'hbase
>>>>>>>>>>>> shell'
>>>>>>>>>>>> on
>>>>>>>>>>>> the machine.  The hbase-site.xml is ...
>>>>>>>>>>>>
>>>>>>>>>>>>    <configuration>
>>>>>>>>>>>>    <property>
>>>>>>>>>>>>    <name>hbase.zookeeper.quorum</**name>
>>>>>>>>>>>>    <value>ip-XX-YYY-Z-QQQ.ec2.**internal</value>
>>>>>>>>>>>>    </property>
>>>>>>>>>>>>    <property>
>>>>>>>>>>>>    <name>hbase.cluster.**distributed</name>
>>>>>>>>>>>>    <value>true</value>
>>>>>>>>>>>>    </property>
>>>>>>>>>>>>    <property>
>>>>>>>>>>>>    <name>hbase.rootdir</name>
>>>>>>>>>>>>    <value>hdfs://localhost/hbase<**/value>
>>>>>>>>>>>>    </property>
>>>>>>>>>>>>    </configuration>
>>>>>>>>>>>>
>>>>>>>>>>>> The client machine hbase-site.xml is...
>>>>>>>>>>>>
>>>>>>>>>>>>    <configuration>
>>>>>>>>>>>>    <property>
>>>>>>>>>>>>    <name>hbase.zookeeper.quorum</**name>
>>>>>>>>>>>>    <value>ip-XX-YYY-Z-QQQ.ec2.**internal.ec2.internal</value>
>>>>>>>>>>>>    <description>Pseudo Distributed
Server</description>
>>>>>>>>>>>>    </property>
>>>>>>>>>>>>    </configuration>
>>>>>>>>>>>>
>>>>>>>>>>>> And when I try 'hbase shell' remotely, I
get this...
>>>>>>>>>>>>
>>>>>>>>>>>>    hbase(main):001:0>            status
'detailed'
>>>>>>>>>>>>
>>>>>>>>>>>>    ERROR: org.apache.hadoop.hbase.**ZooKeeperConnectionException:
>>>>>>>>>>>> HBase
>>>>>>>>>>>>    is able to connect to ZooKeeper but the
connection closes
>>>>>>>>>>>>    immediately. This could be a sign that
the server has too many
>>>>>>>>>>>>    connections (30 is the default). Consider
inspecting your ZK
>>>>>>>>>>>> server
>>>>>>>>>>>>    logs for that error and then make sure
you are reusing
>>>>>>>>>>>>    HBaseConfiguration as often as you can.
See HTable's javadoc
>>>>>>>>>>>> for
>>>>>>>>>>>>    more information.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message