hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billy Pearson" <sa...@pearsonwholesale.com>
Subject bulk load issues with negotiated timeout
Date Mon, 07 May 2012 05:07:24 GMT
I am running test on the bulk load option on hbase I run the map and 
everything runs fine for small data but I tryto load 3-500MB of data and I 
get timed out and it tries to load the data several times
I found the problem just can not find how to fix it how to do you change the 
negotiated timeout = 40000 to something else

I tried to do it in the job config before I load the data to a high number 
to see if it helped
conf.set("zookeeper.session.timeout", "1210000");
but that does not seams to change the 40 sec timer below.

So the question is what option changes the 40 secs and where does it go in 
the conf in zookeeper/hbase/ or hadoop conf file?
also can I change it just for this job or does it have to be changed cluster 
wide?

12/05/06 23:54:23 INFO zookeeper.ZooKeeper: Initiating client connection, 
connectString=node2:2181,node1:2181 sessionTimeout=180000 
watcher=hconnection
12/05/06 23:54:23 INFO zookeeper.ClientCnxn: Opening socket connection to 
server node1/10.0.0.71:2181
12/05/06 23:54:23 INFO zookeeper.ClientCnxn: Socket connection established 
to node1/10.0.0.71:2181, initiating session
12/05/06 23:54:23 INFO zookeeper.ClientCnxn: Session establishment complete 
on server node1/10.0.0.71:2181, sessionid = 0x37100d3f3000f3, negotiated 
timeout = 40000
12/05/06 23:54:24 INFO zookeeper.ZooKeeper: Initiating client connection, 
connectString=node2:2181,node1:2181 sessionTimeout=1210000 
watcher=hconnection
12/05/06 23:54:24 INFO zookeeper.ClientCnxn: Opening socket connection to 
server node2/10.0.0.72:2181
12/05/06 23:54:24 INFO zookeeper.ClientCnxn: Socket connection established 
to node2/10.0.0.72:2181, initiating session
12/05/06 23:54:24 INFO zookeeper.ClientCnxn: Session establishment complete 
on server node2/10.0.0.72:2181, sessionid = 0x137100d3f2600ee, negotiated 
timeout = 40000


code used to load after the mr job is done

conf.set("zookeeper.session.timeout", "1210000");
conf.set("hbase.zookeeper.property.tickTime", "6000");
HTable table = new HTable("Repo");
LoadIncrementalHFiles lh = new LoadIncrementalHFiles(conf);
lh.doBulkLoad(new Path(otherArgs[1]), table);

 



Mime
View raw message