hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: HBase export limit bandwith
Date Wed, 04 Jun 2014 14:06:42 GMT
There have been 20 releases for 0.94 - the major release after 0.92.x

Please consider upgrading your cluster.

Cheers


On Wed, Jun 4, 2014 at 5:59 AM, Damien Hardy <dhardy@viadeoteam.com> wrote:

> Not yet as we a stick with cdh4.1.2 (hbase 0.92.1) for now.
>
> Le 04/06/2014 14:51, Ted Yu a écrit :
> > Can you use ExportSnapshot tool ?
> >
> > ExportSnapshot has option to limit bandwidth. See HBASE-11090.
> > This feature is in the soon-to-be-released 0.98.3
> >
> > See HBASE-11204 as well.
> >
> > Cheers
> >
> > On Jun 4, 2014, at 5:39 AM, Damien Hardy <dhardy@viadeoteam.com> wrote:
> >
> >> Hello,
> >>
> >> We are trying to export HBase table on S3 for backup purpose.
> >> By default export tool run a map per region and we want to limit output
> >> bandwidth on internet (to amazon s3).
> >>
> >> We were thinking in adding some reducer to limit the number of writers
> >> but this is explicitly hardcoded to 0 in Export class
> >> ```
> >>    // No reducers. Just write straight to output files.
> >>    job.setNumReduceTasks(0);
> >> ```
> >>
> >> Is there an other way (propertie?) in hadoop to limit output bandwidth ?
> >>
> >> --
> >> Damien
> >>
>
> --
> Damien HARDY
> IT Infrastructure Architect
> Viadeo - 30 rue de la Victoire - 75009 Paris - France
> PGP : 45D7F89A
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message