hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <tdunn...@maprtech.com>
Subject Re: HBase 0.90.0 region servers dying
Date Wed, 16 Feb 2011 09:00:07 GMT
Are the nodes themselves dying or the region server processes?

What JVM version?

On Wed, Feb 16, 2011 at 12:46 AM, Ryan Rawson <ryanobjc@gmail.com> wrote:

> are your disks filling? are you running into swap? vmstat can help
> diagnose this.
>
> What is 'heavy load'. I pushed a 3 node hbase cluster to 18-24k
> ops/sec and I didn't feel like it hit hard enough.
>
> Also what was the tail of the log looking like? Any "FATAL" or "ERROR"
> strings? Or grep for Exception?
>
> -ryan
>
> On Wed, Feb 16, 2011 at 12:40 AM, Enis Soztutar
> <enis.soz.nutch@gmail.com> wrote:
> > Hi,
> >
> > We have a newly setup a cluster of 5 nodes, each with 16 GB rams. We use
> > HBase 0.90.0 on top of Hadoop from CDH3. When testing HBase under heavy
> load
> > generated bu YCSB, we consistently see region servers dying silently,
> > without any logs or exceptions (not even in system logs). We couldn't
> track
> > down the problem, so we have  tested the same setup on a rackspace
>  cluster
> > with 7 nodes but similar hardware, and we didn't have any problem.
> >
> > We are suspecting a problem with the rams, or motherboards, but all
> memory
> > tests run successfully. I was wondering if anyone had similar problems
> > before and is there anything you suggest to nail down the issue.
> >
> > Thanks,
> > Enis
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message