hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Beaudreault <bbeaudrea...@hubspot.com>
Subject Re: Hbase regionserver.MultiVersionConcurrencyControl Warning
Date Fri, 05 Aug 2016 15:52:41 GMT
I'm also interested in an answer here. We see this from time to time in our
production HBase clusters (non-opentsdb). It seems to be related to
contention under heavy reads or heavy writes. But it's not clear what the
impact is here.

On Fri, Aug 5, 2016 at 5:14 AM Sterfield <sterfield@gmail.com> wrote:

> Hi,
>
> I'm currently testing Hbase 1.2.1 + OpenTSDB. For that, I'm generating a
> high load of HTTP PUT on OpenTSDB, which then writes in Hbase. Currently,
> I'm able to feed 300k data point per seconds, and I'm trying to achieve
> higher speed.
>
> I have also activate JMX on both Master and Region servers in order to grab
> a bunch of metrics from Hbase.
>
> At some point, I'm noticing a bunch of warnings in a region server log file
> :
>
> 2016-08-04 05:12:06,537 WARN
>  [B.defaultRpcServer.handler=21,queue=3,port=16020]
> regionserver.MultiVersionConcurrencyControl: STUCK:
> MultiVersionConcurrencyControl{readPoint=7714719, writePoint=7714778}
> 2016-08-04 05:12:06,538 WARN
>  [B.defaultRpcServer.handler=19,queue=1,port=16020]
> regionserver.MultiVersionConcurrencyControl: STUCK:
> MultiVersionConcurrencyControl{readPoint=7714719, writePoint=7714778}
> 2016-08-04 05:12:06,538 WARN
>  [B.defaultRpcServer.handler=40,queue=4,port=16020]
> regionserver.MultiVersionConcurrencyControl: STUCK:
> MultiVersionConcurrencyControl{readPoint=7714719, writePoint=7714778}
> 2016-08-04 05:12:06,548 WARN
>  [B.defaultRpcServer.handler=0,queue=0,port=16020]
> regionserver.MultiVersionConcurrencyControl: STUCK:
> MultiVersionConcurrencyControl{readPoint=7714719, writePoint=7714778}
> 2016-08-04 05:12:06,549 WARN
>  [B.defaultRpcServer.handler=35,queue=5,port=16020]
> regionserver.MultiVersionConcurrencyControl: STUCK:
> MultiVersionConcurrencyControl{readPoint=7714719, writePoint=7714778}
> 2016-08-04 05:12:06,556 WARN
>  [B.defaultRpcServer.handler=7,queue=1,port=16020]
> regionserver.MultiVersionConcurrencyControl: STUCK:
> MultiVersionConcurrencyControl{readPoint=7714719, writePoint=7714779}
> 2016-08-04 05:12:06,558 WARN
>  [B.defaultRpcServer.handler=20,queue=2,port=16020]
> regionserver.MultiVersionConcurrencyControl: STUCK:
> MultiVersionConcurrencyControl{readPoint=7714719, writePoint=7714779}
> 2016-08-04 05:12:06,559 WARN
>  [B.defaultRpcServer.handler=14,queue=2,port=16020]
> regionserver.MultiVersionConcurrencyControl: STUCK:
> MultiVersionConcurrencyControl{readPoint=7714719, writePoint=7714779}
> 2016-08-04 05:12:06,609 WARN
>  [B.defaultRpcServer.handler=42,queue=0,port=16020]
> regionserver.MultiVersionConcurrencyControl: STUCK:
> MultiVersionConcurrencyControl{readPoint=7714719, writePoint=7714779}
> 2016-08-04 05:12:06,609 WARN
>  [B.defaultRpcServer.handler=32,queue=2,port=16020]
> regionserver.MultiVersionConcurrencyControl: STUCK:
> MultiVersionConcurrencyControl{readPoint=7714719, writePoint=7714779}
> 2016-08-04 05:12:06,619 WARN
>  [B.defaultRpcServer.handler=2,queue=2,port=16020]
> regionserver.MultiVersionConcurrencyControl: STUCK:
> MultiVersionConcurrencyControl{readPoint=7714719, writePoint=7714779}
> 2016-08-04 05:12:06,619 WARN
>  [B.defaultRpcServer.handler=48,queue=0,port=16020]
> regionserver.MultiVersionConcurrencyControl: STUCK:
> MultiVersionConcurrencyControl{readPoint=7714719, writePoint=7714779}
>
> Can someone shed some lights on this ? I've googling + searching the
> documentation, but found no clue on this subject.
>
> Is it a problem ? If so, how can I fix it ?
>
> Thanks a lot for your help,
>
> Guillaume
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message