Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAAknh04uO9adTb8GbQt2s2eaXFpkV=94G_kt=ahSNBsncy7sxw@mail.gmail.com>
References: 
 <CALJs9_2iazkwBwGmvZ=BSTXEUqmk6Hjs4SdYPdapUpBKw-JspA@mail.gmail.com>
	<CALJs9_3WMLKjVPuBNP7vjerZCja8=250jQRKiEbsVwjxE1zz_w@mail.gmail.com>
	<CAA4mzonoihfLO+doeOys1zvxHindF=oSPbr3Bu3BBUp63suhng@mail.gmail.com>
	<CAAknh04uO9adTb8GbQt2s2eaXFpkV=94G_kt=ahSNBsncy7sxw@mail.gmail.com>
Date: Wed, 10 Jun 2015 20:17:09 -0700
Message-ID: 
 <CAAg3a2qhY2Cw_3pPg++wpEbEZJ4m_rC-ujcQ7xm0+TXztqYNww@mail.gmail.com>
Subject: Re: How to monitor and control heavy network traffic for region
 server
From: Vladimir Rodionov <vladrodionov@gmail.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a1140690c4688e30518356dc7

--001a1140690c4688e30518356dc7
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Louis,

What do you mean by "monitor the long scan"? If you need to throttle
network IO during scan, you have to
do it on a client side. Take a look at
org.apache.hadoop.hbase.io.hadoopbackport.ThrottledInputStream
as an example, something similar you will need to implement on top of
ResultScanner - ThrottledResultScanner.

Good idea for improvement, actually.

-Vlad

On Wed, Jun 10, 2015 at 7:30 PM, Louis Hust <louis.hust@gmail.com> wrote:

> hi, Dave,
>
> For now we will not upgrade the version, so if there is something we can
> monitor the long scan for 0.96?
>
> 2015-06-11 2:00 GMT+08:00 Dave Latham <latham@davelink.net>:
>
> > I'm not aware of anything in version 0.96 that will limit the scan for
> > you - you may have to do it in your client yourself.  If you're
> > willing to upgrade, do check out the throttling available in HBase
> > 1.1:
> >
> https://blogs.apache.org/hbase/entry/the_hbase_request_throttling_feature
> >
> >
> >
> > On Wed, Jun 10, 2015 at 4:15 AM, =E5=A8=84=E5=B8=85 <louis.hust.ml@gmai=
l.com> wrote:
> > > any ideas?
> > >
> > > 2015-06-10 16:01 GMT+08:00 =E5=A8=84=E5=B8=85 <louis.hust.ml@gmail.co=
m>:
> > >
> > >> Hi all,
> > >>
> > >> We are using HBASE 0.96 with Hadoop 2.2.0, recently we found there a=
re
> > some
> > >> SCAN operations last for more than 1 hours, which lead to heavy
> network
> > >> traffic, because some data is not
> > >> stored at local data node and the region is very big, about 100G-500=
G=E3=80=82
> > >>
> > >> With heavy network traffic, the region server then can not offer
> service
> > >> cause the NIC is full.
> > >>
> > >> From DBA view point, I want to know if there is some operation for
> HBase
> > >> to limit the large scan or if there is some flow control for SCAN
> > operation
> > >> and how can we monitor the long SCAN operation=E3=80=82
> > >>
> >
>

--001a1140690c4688e30518356dc7--