hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikael Sitruk <mikael.sit...@gmail.com>
Subject Re: warning in log - question
Date Thu, 22 Dec 2011 09:54:55 GMT
JD hi

The system is under heavy load only via hbase (no map reduce running).
Another thing is that we have hard constraints in latencies, therefore if
this point on performance problem i would like to "fix" it. The GC seems
fine too.
I see around 300 message like this in 10 hours of run, approx each 2
minutes intervals.
So now the question - what should i do to reduce the time taken (knowing
that the data to be written will remain the same in volume)?

Regards,
Mikael.S

On Wed, Dec 21, 2011 at 7:44 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> That line means that appending to the log took x ms, we print it out
> when it takes more than 1 second. I'd be worried to see that a lot in
> my logs if I had a realtime-ish system because it would mean that my
> writes are taking seconds.
>
> The usual reasons:
>
>  - Heavy IO, I would expect those lines if I was writing a lot of data
> into HBase or running MR jobs that don't use HBase but do push a lot
> of data to their datanodes.
>  - GC, for example you have a pretty busy system with not a lot of
> memory so it's running long-ish GCs every now and then well if it
> happens during the append operation it may appear that it took x ms,
> but in fact could have been super short but the JVM was blocked for
> some time.
>
> Hope this helps,
>
> J-D
>
> On Wed, Dec 21, 2011 at 1:11 AM, Mikael Sitruk <mikael.sitruk@gmail.com>
> wrote:
> > Hi
> >
> > I have my log with lot of the following message in my hbase logs.
> >
> > 2011-12-21 00:32:05,035 WARN
> org.apache.hadoop.hbase.regionserver.wal.HLog:
> > IPC Server handler 38 on 60020 took 1247 ms appending an edit to hlog;
> > editcount=625, len~=61.4k
> > 2011-12-21 00:32:06,378 WARN
> org.apache.hadoop.hbase.regionserver.wal.HLog:
> > IPC Server handler 33 on 60020 took 1342 ms appending an edit to hlog;
> > editcount=627, len~=61.4k
> >
> > Can you explain the what does it mean? What should i do to remove this
> > warning? Does it have performance impact?
> >
> > Thanks
> > Mikael.S
>



-- 
Mikael.S

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message