hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stack <st...@duboce.net>
Subject Re: failure after importing 42million rows
Date Fri, 25 Jul 2008 18:59:56 GMT
Mark Snow wrote:
> I'm running a hbase data import on 0.1.3. After 42million rows, the import fails with
an RPC timeout exception. I've tried twice- once on a 2 node cluster and once on a 10 node
cluster (ec2 with the same configuration) and it failed both times in the same spot, somewhere
between 42 and 43 million rows. 
Small, medium, or X-large instances?

> Where should I look to debug this?
>
> >From the hbase shell, I can query the table and see the rows have been inserted,
but when I do a 'hadoop dfs -ls' I don't see the /hbase dir I specified, so I'm suspicious
it's not storing the data into dfs, and unsure where it is storing this data.
>   

The $HADOOP_HOME that you are running the 'hadoop dfs -ls' under has in 
its conf file hdfs://domU-12-31-39-00-E9-23:5001/ as the  fs.default.name?

Perhaps 'hadoop  dfs -fs hdfs://domU-12-31-39-00-E9-23:50001/ -lsr 
/hbase' works?

Otherwise, nothing untoward in what you sent in email.  Whats the RPC 
error you're seeing?  Try things like upping your lease periods.    Try 
doubling hbase.regionserver.lease.period and hbase.master.lease.period.  
Are you loading via MR or via a custom script?  If the former, are 
TaskTrackers running on all nodes beside Regionservers and Datanodes?

St.Ack


Mime
View raw message