hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <lhofha...@yahoo.com>
Subject Re: Many scanner opening
Date Thu, 20 Dec 2012 18:51:36 GMT
Cool.

You probably made it less likely that your scanners will scan the same HFile in parallel.

-- Lars



________________________________
 From: Eugeny Morozov <emorozov@griddynamics.com>
To: user@hbase.apache.org; lars hofhansl <lhofhansl@yahoo.com> 
Sent: Thursday, December 20, 2012 2:32 AM
Subject: Re: Many scanner opening
 
Lars,

Cool stuff! Thanks a lot! I'm not sure I can apply the patch, cause we're
using CDH-4.1.1, but increasing size of internal scanner does the trick -
decreased number of scanners.
At least temporarily it's good enough.

Thanks!

On Wed, Dec 19, 2012 at 6:23 AM, lars hofhansl <lhofhansl@yahoo.com> wrote:

> You might have run into HBASE-7336.
> (Not available in any official release, yet)
>
> If you're using 0.94 (and probably 0.92) you can just apply this patch
> (it's save and simple).
>
>
>
> ________________________________
>  From: Eugeny Morozov <emorozov@griddynamics.com>
> To: user@hbase.apache.org
> Sent: Tuesday, December 18, 2012 12:01 AM
> Subject: Many scanner opening
>
> Hello!
>
> We faced an issue recently that the more map tasks are completed, the
> longer it takes to complete one more map task.
>
> In our architecture we have two scanners to read the table. The first one,
> which is called 'outer' scanner is reading table and filter some rowkeys.
> These rowkeys are used as a filter for second scanner - 'internal'. Thus we
> constantly open 'internal' scanner with different filters.
>
> As an additional symptoms we see that our cluster practically does nothing
> - there is no CPU loading, no disk loading, no network, etc. Most of the
> time it means we are waiting on some locks, but I'm not sure.
>
> I would appreciate any ideas or suggestions to understand the case.
> Thank you in advance.
> --
> Evgeny Morozov
> Developer Grid Dynamics
> Skype: morozov.evgeny
> www.griddynamics.com
> emorozov@griddynamics.com
>



-- 
Evgeny Morozov
Developer Grid Dynamics
Skype: morozov.evgeny
www.griddynamics.com
emorozov@griddynamics.com
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message