hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Himanish Kushary <himan...@gmail.com>
Subject Re: Performance degrades on moving from desktop to blade environment
Date Mon, 16 May 2011 20:24:33 GMT
Yes, it is only the HW that was changed . All the configurations are kept at
default from the cloudera installer.

The regionserver logs semms ok.

On Mon, May 16, 2011 at 3:20 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> Ok I see... so the only thing that changed is the HW right? No
> upgrades to a new version? Also could it be possible that you changed
> some configs (or missed them)? BTW counting has a parameter for
> scanner caching, like you would write: count "myTable", CACHE = 1000
>
> and it should stream through your data.
>
> Anything weird in the region server logs?
>
> J-D
>
> On Mon, May 16, 2011 at 12:13 PM, Himanish Kushary <himanish@gmail.com>
> wrote:
> > Thanks for the reply. We ran the TestDFSIO benchmark on both the
> development
> > and production and found the production to be better.The statistics are
> > shown below.
> >
> > But once we bring HBase into the picture things gets reversed :-(
> >
> > The count operation,map-reduces etc becomes less performing on the
> > production box.We are using Pseudo Distribution mode in both the
> development
> > and production servers for both hadoop and hbase.
> >
> > *DEVELOPMENT SERVER*
> >
> > 11/05/15 21:26:26 INFO fs.TestDFSIO: ----- TestDFSIO ----- : write
> > 11/05/15 21:26:26 INFO fs.TestDFSIO:            Date & time: Sun May 15
> > 21:26:26 EDT 2011
> > 11/05/15 21:26:26 INFO fs.TestDFSIO:        Number of files: 10
> > 11/05/15 21:26:26 INFO fs.TestDFSIO: Total MBytes processed: 10000
> > 11/05/15 21:26:26 INFO fs.TestDFSIO:      Throughput mb/sec:
> > 58.09495038691237
> > 11/05/15 21:26:26 INFO fs.TestDFSIO: Average IO rate mb/sec:
> > 59.699485778808594
> > 11/05/15 21:26:26 INFO fs.TestDFSIO:  IO rate std deviation:
> > 10.54547265175703
> > 11/05/15 21:26:26 INFO fs.TestDFSIO:     Test exec time sec: 163.354
> > 11/05/15 21:26:26 INFO fs.TestDFSIO:
> >
> > 11/05/15 21:28:44 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read
> > 11/05/15 21:28:44 INFO fs.TestDFSIO:            Date & time: Sun May 15
> > 21:28:44 EDT 2011
> > 11/05/15 21:28:44 INFO fs.TestDFSIO:        Number of files: 10
> > 11/05/15 21:28:44 INFO fs.TestDFSIO: Total MBytes processed: 10000
> > 11/05/15 21:28:44 INFO fs.TestDFSIO:      Throughput mb/sec:
> > 682.4075337791729
> > 11/05/15 21:28:44 INFO fs.TestDFSIO: Average IO rate mb/sec:
> > 755.5845947265625
> > 11/05/15 21:28:44 INFO fs.TestDFSIO:  IO rate std deviation:
> > 229.60029445080488
> > 11/05/15 21:28:44 INFO fs.TestDFSIO:     Test exec time sec: 63.896
> > 11/05/15 21:28:44 INFO fs.TestDFSIO:
> >
> >
> >
> >
> > *PRODUCTION SERVER*
> >
> > 5/16 01:00:43 INFO fs.TestDFSIO: ----- TestDFSIO ----- : *WRITE
> PERFORMANCE*
> >
> > 11/05/16 01:00:43 INFO fs.TestDFSIO: Date & time: Mon May 16 01:00:43
> > GMT+00:00 2011
> >
> > 11/05/16 01:00:43 INFO fs.TestDFSIO: Number of files: 10
> >
> > 11/05/16 01:00:43 INFO fs.TestDFSIO: Total MBytes processed: 10000
> >
> > 11/05/16 01:00:43 INFO fs.TestDFSIO: Throughput mb/sec: 69.25447557048375
> >
> > 11/05/16 01:00:43 INFO fs.TestDFSIO: Average IO rate mb/sec:
> > 70.06581115722656
> >
> > 11/05/16 01:00:43 INFO fs.TestDFSIO: IO rate std deviation:
> > 7.243961483443693
> >
> > 11/05/16 01:00:43 INFO fs.TestDFSIO: Test exec time sec: 126.896
> >
> >
> > 5/16 01:25:01 INFO fs.TestDFSIO: ----- TestDFSIO ----- : *READ
> PERFORMANCE*
> >
> > 11/05/16 01:25:01 INFO fs.TestDFSIO: Date & time: Mon May 16 01:25:01
> > GMT+00:00 2011
> >
> > 11/05/16 01:25:01 INFO fs.TestDFSIO: Number of files: 10
> >
> > 11/05/16 01:25:01 INFO fs.TestDFSIO: Total MBytes processed: 10000
> >
> > 11/05/16 01:25:01 INFO fs.TestDFSIO: Throughput mb/sec: 1487.20999405116
> >
> > 11/05/16 01:25:01 INFO fs.TestDFSIO: Average IO rate mb/sec:
> > 1525.230712890625
> >
> > 11/05/16 01:25:01 INFO fs.TestDFSIO: IO rate std deviation:
> > 239.54492784268226
> >
>



-- 
Thanks & Regards
Himanish

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message