hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Effect of region size on compaction performance
Date Sun, 23 Mar 2014 02:47:14 GMT
See HBASE-5140 TableInputFormat subclass to allow N number of splits per
region during MR jobs
where there was some unfinished work.

Cheers


On Sat, Mar 22, 2014 at 7:28 PM, David Koch <ogdude@googlemail.com> wrote:

> Hi Ted,
>
> Thank you for your reply. I am aware of the possibility of scanning over
> multiple tables in one M/R however this is not applicable in our case.
>
> Regards,
>
> /David
>
>
> On Sun, Mar 23, 2014 at 3:10 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > David:
> > Have you looked at HBASE-3996 ' Support multiple tables and scanners as
> > input to the mapper in map/reduce jobs' ?
> >
> > Cheers
> >
> >
> > On Sat, Mar 22, 2014 at 6:58 PM, David Koch <ogdude@googlemail.com>
> wrote:
> >
> > > Hello,
> > >
> > > We run M/Rs over several HBase tables at the same time and chose to
> > reduce
> > > region sizes in order to make map tasks faster and improve map-slot
> > > turnaround between the concurrent jobs. However, I am worried many
> > regions
> > > will cause longer overall compactions of the HBase data. Is this the
> > case?
> > >
> > > I am thinking about extending TableInputFormat to override the
> > > 1-map-per-region default policy as an alternative.
> > >
> > > Regards,
> > >
> > > /David
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message