hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joarder KAMAL <joard...@gmail.com>
Subject Re: Confusion with Actual Number of Regions in HBase
Date Thu, 21 Feb 2013 01:39:47 GMT
Hi Ted,

After login into the HBase UI I found two of my region servers were dead.
Don't have any clue why. Master is still up and running for the past 4 days.
Memory is still running pretty low without any workload and two region
servers!!

--------------------------------------------------

root@hdb1-01:/home/ubuntu# jps
1064 NameNode
1578 JobTracker
10770 Jps
2481 HRegionServer
2261 HMaster
1280 DataNode
1769 TaskTracker
2197 HQuorumPeer
1480 SecondaryNameNode
--------------------------------------------------
root@hdb1-02:/home/ubuntu# jps
1492 HRegionServer
1315 TaskTracker
1137 DataNode
3290 Jps
--------------------------------------------------
root@hdb1-03:/home/ubuntu# jps
1527 HRegionServer
1171 DataNode
7475 Jps
1349 TaskTracker

--------------------------------------------------
PID found 2197, 2261 and 2481 for 'hbase'
--------------------------------------------------
root@hdb1-01:/home/ubuntu# ps axo pid,ppid,rss,vsz | grep 2197
 2197     1 52248 1938732
--------------------------------------------------
root@hdb1-01:/home/ubuntu# ps axo pid,ppid,rss,vsz | grep 2261
 2261     1 108724 2007912
--------------------------------------------------
root@hdb1-01:/home/ubuntu# ps axo pid,ppid,rss,vsz | grep 2481
 2481     1 686252 2031376
--------------------------------------------------
root@hdb1-01:/home/ubuntu# free -m
             total       used       free     shared    buffers     cached
Mem:     1652       1616         36          0         51         305

-/+ buffers/cache:       1259        393
Swap:          895          8        887


Thanks a lot again ...


Regards,
Joarder Kamal



On 21 February 2013 11:08, Ted Yu <yuzhihong@gmail.com> wrote:

> What do you see when you use 'ps aux | grep <pid>' command ?
> A sample is shown below:
>
> USER              PID  %CPU %MEM      VSZ    RSS   TT  STAT STARTED
>  TIME COMMAND
> tyu             16468 232.2  1.4  4974136 240528 s002  R+    4:05PM
> 2:11.15
>
> /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/javadoc
> -J-Xmx2048m @options @packages
>
> Please give us the VSZ and RSS readings.
>
> Thanks
>
> On Wed, Feb 20, 2013 at 3:56 PM, Joarder KAMAL <joarderm@gmail.com> wrote:
>
> > Sorry may be I didn't clarify the question properly.
> >
> > Q2. As per the default setting maxHeapMB size is 991, so in saturation
> > condition HBase can't use more RAM (even if it is available, for example
> > the system memory size is 4GB) than this defined size, is it right?
> >
> >
> > Many thanks again...
> >
> > On 21 February 2013 10:48, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> > > I am not familiar with monitoring tools on EC2.
> > >
> > > You can use Ganglia to monitor hadoop / HBase.
> > >
> > > Cheers
> > >
> > > On Wed, Feb 20, 2013 at 3:42 PM, Joarder KAMAL <joarderm@gmail.com>
> > wrote:
> > >
> > > > Hi Ted,
> > > >
> > > > Thanks for the reply.
> > > >
> > > > Q1. Cleared now. Sorry I didn't count the -ROOT- and .META. tables
> > > > Q2. Not clear. Could explain a bit more? I am using the default
> > setting.
> > > >
> > > > Thanks again.
> > > >
> > > >
> > > > On 21 February 2013 10:36, Ted Yu <yuzhihong@gmail.com> wrote:
> > > >
> > > > > bq. but when I click to view the table details it shows there are
5
> > > > > regions.
> > > > >
> > > > > Did you include -ROOT- and .META. tables above ?
> > > > >
> > > > > bq. as you can see the maxHeapMB size is 991
> > > > >
> > > > > Attachment is automatically stripped from your email. FYI
> > > > >
> > > > > On Wed, Feb 20, 2013 at 3:26 PM, Joarder KAMAL <joarderm@gmail.com
> >
> > > > wrote:
> > > > >
> > > > > > Dear All,
> > > > > >
> > > > > > I am currently running a 3 node HBase cluster with Amazon EC2
> small
> > > > > > instance (1CPU, 1.7GB Mem). After running a simple workload
(type
> > A)
> > > > > using
> > > > > > YCSB benchmarking tool, I noticed in the Web UI that the number
> of
> > > > > regions
> > > > > > in the 3 region servers are 7 but when I click to view the table
> > > > details
> > > > > it
> > > > > > shows there are 5 regions. I am a bit confuse. Does the Web
UI
> > > showing
> > > > > > wrong results? How can I check region details from the shell/is
> > there
> > > > any
> > > > > > other command line tools for that? I am using Hbase 0.94.3 (as
it
> > is
> > > > the
> > > > > > latest version compatible with YCSB at this moment) with
> > > Hadoop-1.0.4.
> > > > > >
> > > > > > Another thing, as you can see the maxHeapMB size is 991 (default
> > > > setting)
> > > > > > then whenever I am running workloads the server free memory
> status
> > is
> > > > > > terrible - almost 0MB for 1M/10M transactions (YCSB/Workload-A).
> My
> > > > > > question is if the maxHeapMB is fixed at 1GB then why I am
> loosing
> > > > extra
> > > > > > 700MB memory? How can I monitor cpu/memory/io usage for
> individual
> > > > > cluster
> > > > > > components (HBase/Hadoop/MapRed/ZK, etc.)? Is there any tool
> > > available
> > > > > with
> > > > > > HBase?
> > > > > >
> > > > > > Thanks a lot.
> > > > > >
> > > > > >
> > > > > > [image: Inline images 1]
> > > > > >
> > > > > >
> > > > > > Regards,
> > > > > > Joarder Kamal
> > > > > >
> > > > > > ------------------------------------------
> > > > > > PhD Student
> > > > > > Gippsland School of Information Technology,
> > > > > > Faculty of Information Technology,
> > > > > > MONASH University
> > > > > > Churchill VIC, Australia. 3842
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message