hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert J Berger <rber...@runa.com>
Subject dfs.datanode.max.xcievers > 4k
Date Tue, 27 Sep 2011 06:47:10 GMT
I saw a posting from a year or so ago where Andy said he was running  datanodes with xcievers
set to 10k

Is there any problem doing that? We were running with a 9 regionserver/datanode cluster and
started getting 
java.io.IOException: xceiverCount 4097 exceeds the limit of concurrent xcievers 4096

We had more than 4k regions on most of the nodes. 

We added 4 more nodes but still getting some of the xceiverCount errors every once in a while
(I think from when we are doing backups via the HBase export job).

We're not doing any of the things that could be done to reduce the number of files and regions.
We generally don't have a heave read/write load across the cluster, but we've been doing more
hadoop and backup export jobs than we use to...

We're working on moving to a more modern cluster so I don't want to invest much effort in
the current one, but have to keep it running till we cut over.

Is there any problem to bump up the dfs.datanode.max.xcievers to like 8k or even 10k? 

Can we / should we increase the hbase.hregion.max.filesize to something more than 256M on
a running system?

I presume to change ether we would need to restart the regionservers. Can we do it as a rolling
restart where we change one regionserver at  a time and bring one online before updating the
next? Or does it have to be updated as a cluster?


Robert J Berger - CTO
Runa Inc.
+1 408-838-8896

View raw message