Mailing-List: contact hbase-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-user@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-ID: <415025.76746.qm@web65511.mail.ac4.yahoo.com>
References: <1482.41018.qm@web65508.mail.ac4.yahoo.com>
 <4B057BE2.8000901@purdue.edu>
Date: Thu, 19 Nov 2009 09:28:51 -0800 (PST)
From: Andrew Purtell <apurtell@apache.org>
Subject: Re: EC2  scripts
To: hbase-user@hadoop.apache.org
In-Reply-To: <4B057BE2.8000901@purdue.edu>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="0-623692163-1258651731=:76746"

--0-623692163-1258651731=:76746
Content-Type: text/plain; charset=us-ascii

So it seems the slaves are not checking in with the master for some reason. 

Can you provide the log content from the slave instances in /mnt/hbase/logs/*.log ? 

What is the output of 'hadoop dfsadmin -report'? 

Thanks for trying this out Naresh. 

   - Andy


________________________________
From: Naresh Rapolu <nrapolu@purdue.edu>
To: hbase-user@hadoop.apache.org
Sent: Thu, November 19, 2009 9:09:54 AM
Subject: Re: EC2  scripts

Thanks Andrew,

We`ve made a private instance with our keys,  by altering your public hbase AMI and used it to launch the cluster instances.
We are now able to do  "ssh slave" from master.  But, still the problem persists.  From the hbase shell of master,  "status 'simple' " returns  0 live servers.
Also, while creating a new table from the hbase shell, the following  error pops up.

hbase(main):001:0> create 'dummy', 'dummyfamily'
NativeException: org.apache.hadoop.hbase.master.NotAllMetaRegionsOnlineException: org.apache.hadoop.hbase.master.NotAllMetaRegionsOnlineException
       at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:750)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:597)
       at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:648)
       at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)

       from sun/reflect/NativeConstructorAccessorImpl.java:-2:in `newInstance0'
       from sun/reflect/NativeConstructorAccessorImpl.java:39:in `newInstance'
       from sun/reflect/DelegatingConstructorAccessorImpl.java:27:in `newInstance'
       from java/lang/reflect/Constructor.java:513:in `newInstance'
       from org/apache/hadoop/hbase/RemoteExceptionHandler.java:94:in `decodeRemoteException'
       from org/apache/hadoop/hbase/client/HBaseAdmin.java:208:in `createTableAsync'
       from org/apache/hadoop/hbase/client/HBaseAdmin.java:165:in `createTable'
      -- We`ve checked and ensured that all slaves are online with  HRegionServer  daemons. 
Btw,  Hadoop is working well,  we are able to put in large files into the dfs.   Problem seems to be with Hbase alone.

Let me know if you need any more log messages.

Thanks,
Naresh Rapolu.

Andrew Purtell wrote:
> 1) How long did you wait before logging on to the master and checking status? I bet if you checked the slave instances status at that time (e.g. using elasticfox) they were still pending startup. The slaves are started last and EC2 may not start them right away nor all at once.
> 
> 2) The way private keys work with EC2 instances you must supply the EC2 private key using -i unless you also use this as your default private key, set up the appropriate config (man 5 ssh_config), or build your own private instance with your default key already installed. 
> On Wed Nov 18th, 2009 9:49 PM PST Naresh Rapolu wrote:
> 
>  
>> Hello ,
>> 
>> We were trying to use the  src/contrib/ec2  scripts.  All the instances started well,  but from  hbase shell, on the master ,  a   " status 'simple' "   listed  0  live servers. Also , we are not able to ssh into any of the slave instances from the master,  without using  " shh -i  private_key.pem  ".   (ie)  ssh is not configured to be used without a password and with automatic keys.   I guess, the instances still maintain the creator`s private key.   Did they work for any others, apart from the creator ?   Even the  JIRA page shows that  it didnt completely work  for  St.Ack.
>> 
>> Thanks,
>> Naresh Rapolu
>>    
> 
> 
> 
>        


--0-623692163-1258651731=:76746--