Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of yuzhihong@gmail.com designates
 209.85.160.49 as permitted sender)
References: <538F139E.6050305@viadeoteam.com>
Mime-Version: 1.0 (1.0)
In-Reply-To: <538F139E.6050305@viadeoteam.com>
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-Id: <22C297D9-D7F5-4E69-B8A7-8C945A2A326D@gmail.com>
Cc: user <user@hbase.apache.org>
From: Ted Yu <yuzhihong@gmail.com>
Subject: Re: HBase export limit bandwith
Date: Wed, 4 Jun 2014 05:51:52 -0700
To: "user@hbase.apache.org" <user@hbase.apache.org>

Can you use ExportSnapshot tool ?

ExportSnapshot has option to limit bandwidth. See HBASE-11090. 
This feature is in the soon-to-be-released 0.98.3

See HBASE-11204 as well. 

Cheers

On Jun 4, 2014, at 5:39 AM, Damien Hardy <dhardy@viadeoteam.com> wrote:

> Hello,
> 
> We are trying to export HBase table on S3 for backup purpose.
> By default export tool run a map per region and we want to limit output
> bandwidth on internet (to amazon s3).
> 
> We were thinking in adding some reducer to limit the number of writers
> but this is explicitly hardcoded to 0 in Export class
> ```
>    // No reducers. Just write straight to output files.
>    job.setNumReduceTasks(0);
> ```
> 
> Is there an other way (propertie?) in hadoop to limit output bandwidth ?
> 
> -- 
> Damien
>